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ABSTRACT 
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This  report  consist  of  two  parts.  In  part  one,  a  time  modified 
autoregressive  model  for  interframe  image  coding  is  presented.  This 
method  is  compared  with  previous  work  in  the  field  of  interframe  image 
coding  and  it  is  shown  that  substantial  simplifications  occur  when  the 
nearest  integer  displacement  is  taken  into  account.  It  is  demonstrated 
that  when  the  between  frame  noise  is  minimal  and  the  motion  is  pure 
translation  or  can  be  modelled  by  translation,  enough  information  can 
be  extracted  from  the  predictor  coefficients  to  determine  the  non¬ 
integer  displacement  with  small  error. 

In  part  two,  a  new  concept  for  examining  shapes  as  vectors  in  a 
shape  space  is  described.  The  shape  space  is  defined  in  terms  of  its 
properties  and  the  importance  of  the  independence  of  the  size  variable 
to  the  shape  vectors,  defined  on  this  shape  space,  is  stressed.  Also, 
two  theorems  helpful  in  the  process  of  comparing  partial  shapes  to 
the  complete  shape  are  stated  and  proved.  A  new  method  for  detecting 
the  points  on  a  shape  which  appear  to  dominate  visual  perception  is 
described.  This  method,  called  the  Adaptive  Line  of  Sight  Method 
detects  the  dominant  points  on  a  shape  even  though  they  do  not  always 
occur  on  points  of  high  curvature.  With  this  method,  the  critical 
points,  or  dominant  points,  of  the  shape  that  are  determined  are 
based  on  a  set  of  coordinate  axes  that  are  dependent  on  the  shape 
itself.  Therefore,  the  points  determined  are  independent  of  size, 
rotation,  or  relative  displacement.  ^_r--  _  _  _ - - — - 

The  Line  of  Sight  of  a  point  concept  is  also  introduced  and  sub- 


sequently  utilized  to  extract  features  from  a  shape.  These  features 
are  then  compared  to  the  features  of  other  shapes  by  a  syntactic 
procedure  for  the  purpose  of  recognizing  whether  a  slope  is  a  partial 
shape  or  is  a  shape  in  its  own  right.  It  is  demonstrated  that  the 
feature  vectors  determined  by  this  procedure  are  independent  of 
size,  rotation,  and  displacement.  The  results  of  applying  these 
techniques  to  actual  shapes  are  demonstrated  and  discussed. 
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PROBLEM  STATEMENT 


Many  scientific  disciplines  have  sought  solutions  for 
questions  associated  with  motion.  What  is  motion?  How  is 
motion  defined?  How  is  motion  interpreted?  These  are  just 
a  few  of  the  endless  number  of  questions  that  arise  when 
dealing  with  the  topic  of  motion.  Psychologists  look  to  the 
internal  thought  processes  to  formulate  theories  about  human 
interpretation  of  motion  through  visual  stimulation. 
Physiologists  on  the  other  hand  are  concerned  with  what 
biological  processes  are  necessary  for  the  generation  of  the 
synaptic  signals  associated  with  the  sensations  of  vision 
and  motion.  Engineers  then  try  to  simulate  the  processes  of 
biological  vision  in  machines. 

From  research  carried  out  in  the  above  disciplines,  it 
has  been  found  that  the  human  visual  system  is  a  very 
complex  and  complicated  network  of  biological  subunits. 
Some  examples  of  these  subunits  are  the  light  receptors, 
i.e.  the  rods  and  cones,  the  optic  nerve  for  visual 
transmission,  and  the  brain  with  associated  memory  for 
interpretation.  Each  of  the  subunits  themselves  constitute 
a  very  complex  system.  So  it  is  no  suprise  to  find  that 
when  the  human  visual  system  is  modeled  by  hardware  and 
software  that  the  non-biological  visual  system  will  also  be 
a  very  comf  ex  and  '  replicated  set  of  subunits. 


The  research  effort  detailed  within  deals  with  one  very 
small  subunit  of  the  visual  process,  namely  displacement  or 
motion  detection  and  estimation. 

DISPLACEMENT  AND  MOTION 

Simply  stated,  motion  is  defined  to  be  a  time  series  of 
spatial  displacements.  That  is,  in  order  for  motion  to  be 
perceived,  time  must  pass.  If  artificial  vision  and 
intelligence  is  to  ever  become  a  reality,  then  a 
sufficiently  good  mathematical  model  for  motion  will  have  to 
be  employed.  For  this  reason  and  the  fact  that  memory  space 
will  always  be  limited,  the  vision  system  for  motion  should 
be  based  on  some  time  adaptive  displacement  algorithm. 

The  problem  then  can  be  stated:  Find  a  method  to 
determine  spatial  displacement  from  an  image  sequence  such 
that  an  estimate  of  the  direction  and  magnitude  of  any 
detected  motion  can  be  made. 

The  problem  statement  is  simple  enough,  but  the  effort  is 
complicated  by  many  factors.  One  of  these  factors  involves 
object-background  and  object-foreground  interaction.  For 
example,  if  an  object  in  the  input  frame  moves  in  such  a  way 
so  as  to  uncover  some  background,  complications  will  arise 
in  that  the  new  information  now  consists  not  only  of  the 
object  motion  but  also  in  the  new  background  that  is 
uncovered.  The  system  must  have  the  capacity  to  flag  the 
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difference  between  the  moving  target  and  the  non-moving 
background.  Another  problem  simpler  in  scope  than  the 
above,  but  just  as  important,  involves  the  loss  of  moving 
objects  and  the  addition  of  new  moving  objects  from  outside 
the  field  of  view.  Again  the  system  should  be  able  to 
detect  and  track  these  new  moving  objects  and  discard  the 
exiting  objects. 

Once  the  motion  has  been  identified,  there  will  be  many 
i  ses  for  the  information  thus  provided.  Visual  tracking  of 
moving  objects  will  then  be  possible  as  well  as  target 
trajectory  prediction.  The  displacement  vector  need  not  be 
used  strictly  for  motion  related  studies  but  may  also  be 
used  in  areas  of  data  compression,  remotely  piloted  vehicle 
control,  and  industrial  manufacturing.  With  the  detection 
and  interpretation  of  motion,  a  very  important  step  toward 
artificial  vision  will  have  been  obtained. 

Possible  solutions  to  the  problem  that  seem  non-tractable 
at  present  may  in  the  near  future  be  made  possible  due  to 
the  advances  in  VLSI  technology,  software  development, 
parallel  processing  and  electo-optical  systems.  So  even 
though  the  process  may  look  overly  complicated  and  slow  at 
present  there  may  yet  be  hope  in  the  future. 
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PREVIOUS  WORK  IN  THE  FIELD  OF  MOTION 


There  has  been  relatively  little  work  in  the  area  of 
machine  motion  analysis  until  very  recently  when  the 
required  hardware  and  software  became  available.  The  work 
has  concentrated  in  the  areas  of  displacement  estimation  and 
interframe  image  coding.  One  of  the  earliest,  and  perhaps 
simplest,  methods  used  for  motion  detection  was  simple  image 
frame  differencing.  That  is,  subtract  the  previous  frame, 
pixel  by  pixel,  from  the  current  frame  and  flag  as  motion 
any  difference  greater  than  some  set  threshold. 

M ( i , j , t)  =  ABS [ I ( i  ,  j  ,  t)  -  I (i , j , t- r) ]  (1) 

Motion  will  be  defined  whenever  M(i,j,t)  is  greater  than 
some  threshold.  Although  very  simple,  the  method  does  show 
good  results  for  a  very  limited  class  of  simple  images,  but 
this  method  has  many  drawbacks  that  will  limit  its 
usefulness.  First,  the  output  is  very  sensitive  to  noise  in 
the  input  images  because  it  is  a  differentiating  type 
process.  Also  any  camera  motion  between  image  frames  will 
translate  into  motion  at  every  pixel.  Finally,  no 
information  is  available  pertaining  to  the  direction  or 
magnitude  of  the  motion. 

Many  other  methods  have  been  employed  since  the  first 
frame  differencing  techniques.  A  method  by  Price,  Snyder, 


and  Rajala  [74]  uses  a  Four ier-Domain  filter  based  on  a 


model  of  the  human  visual  system  to  detect  motion.  The 
model  divides  visual  perception  into  two  distinct  channels. 
The  first,  their  so  called  x  channel,  is  a  temporal  low  pass 
and  spatial  band  pass  channel.  This  channel  processes  the 
information  contained  in  two  dimensional  patterns  with  high 
spatial  resolution  but  fairly  low  temporal  dependance.  They 
believe  that  it  is  this  channel  that  is  responsible  for 
objects  with  structural  complexity  but  with  little  or  no 
motion.  The  other,  so  called  y  channel,  is  just  the 
opposite.  Its  characteristics  are  spatial  low  pass  and 
temporal  bandpass.  This  channel,  they  claim,  conveys  the 
information  of  objects  with  high  temporal  dependance  and  low 
spatial  resolution.  It  is  this  y  channel  that  is  used  for 
the  detection  of  motion. 

Stuller,  Netravali  and  Robbins  of  Bell  Laboratories  start 
from  a  completely  different  point  of  view  for  motion. 
First,  the  end  product  of  their  work  is  data  compression  and 
not  tracking,  although  it  could  be  modified  for  such.  The 
model  is  used  for  normal  televison  data  where  adjacent  scan 
rows  are  scanned  by  an  interleaved  method.  In  the  first 
method  of  pel-recursive  displacement  estimation  by  Robbins 
and  Netravali  [78] ,  the  image  model  for  pure  translation 
with  no  background  is  given  in  equation  2. 

I(Xk,t)  =  I(Xk  -  D,t  - t  )  (2) 
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Where:  I(Xk,t)  is  the  intensity  of  the  image  at  the  spatial 
location  Xk  and  time  t.  KX^  -  D,t  -t  )  is  the  intensity  of 
the  image  at  the  spatial  location  Xk  adjusted  by  the 
displacement  D  at  the  previous  time  t-  *.  They  next  define 
a  displaced  frame  difference  term  as 

DFD(Xk,D)  =  I(Xk,t)  -  I(Xk  -  D,t  -  t).  (3) 

An  attempt  is  made  to  minimize  this  difference  with  a 
steepest  descent  algorithm  with  the  form  of  equation  4. 

Dk+1  "  *>k  ~  U/2)«V§  [DFD(Xk,Dk)  ]2  (4) 

Where  iis  a  gain  term  and  VD^  is  a  two-dimensional  gradient 
operator  with  respect  to  Dk  Simplifying  yields, 

Dk+i  -  Dk  -  <  DFD(Xk,Dk)  V  I(Xk  -  Dk,t  -  r  ) .  (5) 

Following  this  same  model,  Netravali  and  Stuller  [67] 
formulated  a  method  for  interframe  coding  termed  coefficient 
recursive  estimation.  It  is  an  extension  of  pel-recursive 
with  the  further  addition  of  a  unitary  transform.  The 
methods  are  similar,  but  now  the  image  is  broken  up  into 
rectangular  blocks  of  size  Nr  rows  by  Nc  columns.  Each 
element  is  then  multiplied  by  the  appropriate  transformation 
vector.  The  blocks  are  then  changed  into  a  column  vector  by 
column  scanning  the  transformed  block.  They  define  the  nth 
coefficient  of  the  qth  block  of  the  transformed  image  to  be, 

cn(q)  -  IT(Xq,t)*n  (6) 
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and  for  the  estimated  displaced  frame, 

cn(q,6)  =  IT(Xq  -  D,  t  -r)*n  (7) 

The  comparable  term  for  the  previous  method's  displaced 
frame  difference  is  the  coefficient  prediction  error  en(q,D) 
and  is  given  by  equation  8. 

en(q,D)  =  [I(Xq/t)  -  I{Xq  -  D,t  -  r  )]T<t>n  (8) 

The  minimization  is  over  the  squared  prediction  error  by  a 
steepest  descent  iteration  of  the  form  in  equation  9. 

VlM  *  °n«J>  -«/2>V6n,q)^<9.V<i>>  (9) 

or  in  a  simpler  form, 

Dn+i(q)  -  Dn(q)  - « en(q,Dn(q) ) Gn(q) .  (10) 

Gn(q)  is  defined  to  be  the  coefficient  gradient  vector. 

Gn(q)  =  [  V  (IT(Xq  -  Dn(q) ,t  -  r ) ] *n  (11) 

When  going  to  the  next  block  the  initial  displacement 
estimate  is  set  to  the  final  estimate  of  the  previous  block. 

D0(q)  =  DN  „  i(q-l)  (12) 

JL  V* 

When  used  for  motion  compensated  interframe  hybrid  transform 
DPCM  coding,  the  coder  transmits  a  quantized  version  of  the 
coefficient  prediction  error  whenever  it  exceeds  some 
threshold.  This  allows  the  reciever  to  update  the  estimate 
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of  the  displacement  and  also  correct  the  prediction 
coefficients.  Figures  1  and  2  point  out  the  differences 
between  a  normal  hybrid  transform-DPCM  coder-decoder  pair, 
figure  1,  and  the  motion  compensated  hybrid  transform-DPCM 
coder-decoder  pair,  given  in  figure  2. 


Figure  1.  Normal  Hybrid  Transform-DPCM  Coding 


Coding 


8 


Results  of  this  and  the  earlier  pel-recursive  method  are 
given  in  figures  4  through  11  for  various  values  of  gain. 
The  test  image  was  a  radially  decaying  cosine  function  of 
radius  60  and  peak-to-peak  amplitude  of  220  at  the  center 
and  130  at  the  cir curierence .  The  period  also  decreased 
radially  starting  with  a  period  of  20  pixels  at  the  center 
and  ending  with  10  pixels  at  the  edge.  The  equation  used  to 
generate  the  test  image  is  as  follows. 

I  (R)  =  100exp  (-0 .01R)  cos  (2rrR/P)  +  128  (13) 

The  displace  image  was  displace  in  the  x  direction  2  pixel 
between  time  frames.  Figure  3  is  a  picture  of  the  actual 
data  used. 


Figure  3.  Radially  decaying  Cosine  Function. 
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DISPLACEMENT  ERROR 


PEL  RECURSIVE 


DISPLACEMENT  ESTIMATION 


Figure  4.  Pel  Recursive  Displacement  Estimation:'. 


PEL  RECURSIVE 


Figure  5.  Pel  Recursive  Displacement  Estimation 


COEFFICIENT  RECURSIVE 


Figure  8.  Coefficient  Recursive  Displacement  Estimation. 


COEFFICIENT  RECURSIVE 


COEFFICIENT  RECURSIVE 


The  data  compression  that  is  achieved  by  the  system  can 


be  attributed  to  two  major  characteristics.  The  first  is 
due  to  the  redundancy  removal  that  is  obtained  by  the 
prediction  process  and  the  second  is  due  to  the  fact  that 
many  of  the  transform  coefficients  can  be  grossly  quantized 
or  completely  neglected  with  acceptable  results. 

One  of  the  major  problems  of  coefficient  recursive 
estimation  is  that  it  is  very  scene  dependent.  In  one  scene 
the  displacement  may  converge  in  as  few  as  4  or  5  iterations 
where  for  others  it  may  fail  to  converge  at  all.  Another 
problem  may  prove  to  be  the  choice  of  a  good  value  for  the 
gain  factor  epsilon.  As  figures  4  through  11  verify,  the 
choice  of  the  gain  factor  plays  an  important  role  in  the 
proper  convergence  of  the  algorithms. 


PREDICTION  COEFFICIENT  ENERGY  CONCENTRATION 

MODEL 

Recalling  again  what  the  mathematical  system  is  to  perform 
and  what  the  restrictions  and  fidelity  criterion  will  be: 

1.  Be  able  to  determine  if  motion  has  occured  in  the 
frame  and  if  so  where  it  has  occured.  Take  the 
information  provided  about  the  motion  and  be  able  to 
track  a  target  or  recombine  the  needed  data  in  such  a 
way  so  as  to  form  a  good  replica  of  the  original 
input  image. 

2.  Minimize  the  data  required  to  update  the  system  from 
one  image  frame  to  the  next,  that  is  achieve  a  fair 
amount  of  data  compression  while  at  the  same  time 
keeping  the  fidelity  sufficiently  high. 

3.  As  a  fidelity  criterion  or  restriction,  minimize  the 
prediction  error  of  both  the  displacement  estimate 
and  the  reconstructed  image  estimate. 

Working  within  these  guidelines  it  can  be  seen  tnat 
coefficient  recursive  displacement  has  met  them  all,  but 
perhaps  not  to  the  extent  that  may  be  possible.  The 
coefficient  energy  concentration  model  uses  a  similar  motion 
model  with  some  major  modifications  in  the  solution.  The 
previous  system  exploited  some  of  the  available  correlation 
of  a  two  dimensional  image,  but  by  no  means  all.  The 
correlation  is  not  only  available  within  the  spatial  domain, 
but  also  over  the  temporal  domain.  For  maximum  data 
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compression  both  spatial  and  temporal  correlation  should  be 
exploited. 

The  image  motion  model  is  as  follows.  For  the  no 
background  noiseless  case  define  the  current  frame  pixel 
intensity  at  the  spatial  location  Xk  to  be  a  linear 
combination  of  pixels  in  the  previous  frame  as  shown  in 
figure  12.  The  double  crosshatched  area  represents  a  pixel 
in  the  current  frame  and  the  single  crosshatched  area 
represents  the  summation  neighborhood  in  the  previous  frame. 
The  summation  neighborhood  is  the  area  in  the  previous  frame 
whose  pixel  values  are  to  be  used  for  the  prediction  of  the 
current  frame  pixel  value. 

I(Xk,t)  a(Yn)I(Xk,t  -r)  (14) 

Yn 


J 


'1 


Figure  12.  Two  Dimensional  Previous  Pixel  Neighborhood. 
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Rewriting  into  the  two  spatial  coordinates  x  and  y 

yields, 

r  j 

I(x,y,t)  =^^a(i,j)I(x  -  i,y  -  j,t  -r)  .  (15) 

-I  -j 

For  two  dimension  non-separable  motion  this  would  require  a 
matrix  of  (21+1)  by  (2J+1)  prediction  coefficients  for  each 
pixel  and  hence  little  if  any  data  compression  could  be 
obtained. 

As  stated  earlier,  it  would  be  advantageous  to  capitalize 
on  both  the  spatial  and  temporal  correlation  that  exists  in 
most  image  data.  In  this  method  the  spatial  correlation  is 
exploited  as  a  result  of  a  linear  combination  being  taken 
within  a  given  time  frame,  and  the  temporal  correlation  is 
taken  advantage  of  when  the  prediction  is  made  from  the  past 
frame  to  the  present. 

Looking  at  the  usual  2  dimensional  autoregressive  model, 
the  previously  defined  model  can  be  seen  to  be  similar  in 
many  respects.  The  form  is  the  same  with  the  deletion  of 
the  prediction  over  time  frames  and  the  previos  frame  pixel 
neighborhood  is  different. 

I(if  j)  -  Jglk^UKi-k^HJ  +  w(i,  j)  (16) 

The  values  for  i  and  j  are  allowed  to  vary  over  the  picture 
size,  the  values  for  k  and  1  are  then  able  vary  to  take  into 
account  values  above  the  current  pixel  and  to  the  left  in 
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the  same  line.  Figure  13  shows  the  pixel  neighborhood  that 
may  be  used  as  in  relation  to  that  in  figure  12. 


Figure  13. 


Current  Frame  Pixel  Neighborhood. 


In  equation  16  the  a(k,l)  matrix  contains  the  regression 
coefficients  and  w(i,j)  then  contains  the  error  terms.  Note 
that  this  is  only  able  to  take  advantage  of  pixel  values  on 
one  side  of  the  current  point,  that  is  the  values  above  it, 
as  is  shown  in  figure  13.  Clearly  this  does  not  exploit  the 
data  to  the  fullest.  Recalling  that  the  model  is  normally 
used  with  a  scanning  system  where  the  values  to  the  right 
and  below  are  not  yet  know  by  the  reciever.  Assuming  full 
knowledge  of  the  image  at  the  previous  time  frame,  a 
prediction  over  the  time  boundary  yield  the  following  form. 

I(i,j,t)  -  ^^a(k,l)I(i-k,j-l,t-r)  +  w(i,j)  (17) 

-K  -L 
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This  in  the  sequel  be  termed  the  time  modified  2  dimensional 
autoregressive  model. 

Noting  what  the  physical  implications  are  in  relations  to 
the  regression  problem,  it  can  be  seen  that  the  coefficients 
are  used  to  perform  a  translation  and  hence  the  model  can  be 
simplified.  Assuming  the  image  fields  can  be  sampled 
infinitely  fast  such  that  they  now  become  continuous 
functions,  the  regression  coefficient  matrix  will  translate 
to  a  delta  function  such  that  the  above  equation  can  be 
written  as, 

I(x,y,t)  =  5 (Ax )*I(x,y,t-r)  +  E(x,y)  (18) 

where  the  function  *  defines  the  convolution  operator.  Now 

the  problem  is  to  determine  the  values  for  A  and  A  or  where 

X  Y 

the  delta  function  is  located.  Here  as  in  the  previous 
methods,  the  theory  works  exactly  only  for  the  pure 
translation  no  background  case,  but  approximations  can  be 
made  and  steps  taken  to  improve  its  usefulness  for  non-ideal 
situations. 

DETERMINING  MAXIMUM  ENERGY  CONCENTRATION 

Noting  that  if  no  noise  exists  then 

I  (x, y,t)  =  5  ;Ax/Ay)*Kx,y,t-r)  (19) 
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should  hold.  Remembering  that  the  sampled  version  I(i,j,t) 
is  available,  the  above  equation  in  discrete  form  becomes, 

I ( i, j , t)  =  5 (A  ,A  ) *  I(i,j,t-r) .  (20) 

X  Y 

Where  the  8(A  »A  )  may  not  fall  on  the  sampling  points  and 

X  Y 

hence  amounts  to  convolving  a  continuous  function  with  a 
discrete  function.  The  countinuous  function  is  simply  a 
delta  function  at  some  displacement.  The  displacement  of 
the  delta  function  from  the  origin  determines  the  image 
displacement  from  frame  to  frame.  The  problem  is  to  solve 
for  the  location  of  this  delta  function.  The  method  used  to 
find  the  location  of  the  delta  function  involves  a  two  step 
process.  First  the  integer  part  of  the  displacement  is 
found  via  the  use  of  a  similarity  metric  and  the  non-integer 
part  by  a  four  coefficient  prediction  process. 

THE  SIMILARITY  METRIC  AND  THE  NON-INTEGER 
PORTION  OF  THE  DISPLACEMENT 

The  similarity  metric  is  used  to  find  the  best  fit  of  the 
current  frame  image  block  with  the  previous  image  frame. 
This  generates  the  nearest  integer  pixel  displacement  for 
the  current  block.  The  similarity  metric  that  is  used  is 
based  upon  a  non-linear  combination  of  the  current  frame 
block  and  the  previous  frame  block.  A  few  different 
combinations  have  been  tried  with  no  real  advantages  or 
disadvantages  exhibited  by  any  one.  One  involves  keeping 
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track  of  the  sum  of  the  absolute  values  of  the  differences 
for  various  values  of  shift. 

P 

M(i,j)  =ZSBS[I(m"i'n-jft-r)  -  I (m,n, t) ]  (21) 

-P  -P 

The  values  for  i  and  j  are  set  so  as  to  vary  over  what  is 
assumed  to  be  the  maximum  displacement  from  frame  to  frame. 
The  estimate  for  the  integer  displacement  is  defined  to  be 
the  point  which  minimizes  the  metric  M(i,j).  One  of  the 
other  possible  metrics  involves  the  sums  of  the  squares  of 
the  differences  for  various  shifts. 

P  P 

M(i,j)  =>^JI(m-i,n-j,t-r)  -  I(m,n,t)]2  (22) 

-P  -P 

Equation  22  offers  the  advantage  of  penalizing  the  metric 
greater  for  larger  errors  than  for  a  number  of  smaller 
errors.  Here  again  the  minimum  of  the  metric  defins  the 
estimate  of  the  integer  displacement.  The  metric  that  has 
proved  to  be  the  most  benificial  to  this  point  is  based  on  a 
counting  procedure.  As  in  the  two  above  equations  either 
the  absolute  value  or  square  of  the  difference  can  be  used. 
Further,  a  two  value  limiting  function  is  define  as  L(x,y). 
Where  L(x,y)  is  define  to  be  1  if  x<y  and  0  if  x>y.  This 
combined  with  the  two  equations  yields  a  counting 
arrangement  as  follows. 

f5,  p 

M(i,  j)  =  X2L(x'  ABS  [  I  (m-i ,  n- j ,  t-  r  )  -  I(m,n,t)J) 

-P^P  (23) 
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At  this  point  the  value  for  x  is  fairly  arbitrary,  but 
somewhat  loosely  related  to  the  variance  of  the  data  noise 
and  may  even  be  a  variable  itself  based  on  the  data.  Again 
as  before,  the  minimum  value  of  the  metric  M(i,j)  will  give 
the  estimate  of  the  integer  part  of  the  displacement. 

Once  an  estimate  has  been  determined  for  the  integer 
portion  of  the  displacement  the  problem  then  remains  to 
determine  where  the  delta  function  is  in  relation  to  the 
integer  displacement.  Figure  14  shows  the  4  possible 
quadrants  about  the  obtained  integer  displacement.  For  the 
noiseless  case  where  only  pure  translation  of  the  image 
field  occurs,  a  similarity  value  indicating  a  perfect  match 
will  exist  in  one  of  the  4  quadrants. 


Figure  14. 
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The  determination  of  which  quadrant  is  based  on  the  values 
generated  for  the  similarity  matrix.  Given  that  the 
similarity  metric  is  a  difference  operator  of  some  sort,  the 
minimum  indicates  the  quadrant  containing  the  delta 
function. 


The  Minimum  quadrant  can  be  found  by  summing  the  values 
of  the  metric  defining  each  quadrant  as  is  shown  in  figure 


15. 
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Q ( 1 )-H+A+B 
Q (2 )-B+C+D 
Q(3)-D+E+F 
Q (4 )-F+G+H 


Figure  15. 

LOCATION  FOR  NON-INTEGER 
PORTION  OF  DISPLACEMENT 

Taking  the  minimum  of  Q ( 1 )  -  Q(4)  indicates  the  correct 
quadrant.  This  only  narrows  down  the  possibilities  for  the 
non-integer  part,  the  actual  calculations  still  remain. 
This  is  solved  by  assigning  a  coefficient  to  each  corner  of 
the  indicated  quadrant  and  solving  for  its  value  in  a 
regression  analysis.  Each  coefficient  is  multiplied  by  the 
corresponding  previous  image  pixel  value,  the  four  are 
summed  and  set  equal  to  the  current  image  value  as  is 
indicated  in  equation  24. 
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I(m,n,t) 


1  1 

*  £  £a(i, j) I (m-dx-i,n-dy- j , t- r  ) 

i=*0  j-0 


Where  dx  and  dy  indicate  the  integer  X  and  Y  displacement 
estimates  respectively.  Note  that  this  is  very  similar  to 
what  was  defined  to  be  the  two  dimensional  time  modified 
autoregressive  model.  The  only  modifications  are  the 
summations  have  been  narrowed  to  a  single  cell  and  the 
addition  of  the  estimated  integer  displacement  values  dx  and 
dy. 


The  problem  then  remains  to  find  values  for  the  predictor 
coefficients  or  a  matrix.  The  solution  is  arrived  at 
through  the  use  of  the  normal  regression  approach.  The 
problem  can  be  further  specified  by  considering  it  a  two 
dimensional  autoregressive  model.  The  usual  linear  first 
order  regression  problem  can  be  written. 

Yi  =  B0  +  B1Xi  +  ei  (25) 

This  then  tries  to  fit  a  straight  line  through  the  data  for 
prediction  purposes.  The  parameters  of  the  model  u.re  B0  and 
Bj  ,the  independent  variable  is  X  and  the  dependent  variable 
is  Y.  Values  for  B0,  B^ ,  and  e  are  unknown  and  hence  the 
problem  is  to  determine  estimates  for  these  values.  Given 
that  b0  is  the  estimate  for  B0  and  bj^  likewise  for  B^  then 
the  predictor  equation  can  be  written  as  follows. 

Y  -  b0  +  b2X  (26) 
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To  find  the  estimates  for  b0  and  the  method  of  least 
squares  is  used,  that  is  minimize  the  sum  of  the  squares  of 
the  error  term  with  respect  to  B0  and  Bj_. 

SSE  *  (Yi  -  B0  -  B^)  (27) 


Find  the  minimum  of  SSE  with  respect  to  B0  and  B^ .  This  is 
accomplished  by  taking  the  partials  and  setting  equal  to 
zero. 


3  SSE 
a  B0 


-  ~  B0  "  BlXi)  “  0 


(28) 


and 


f-f®  -  -2lxi(Yi  -  Ba  '  Bixi>  *  • 


(29) 


i*l 


From  these  the  estimate  b^  for  B^  is  given  by, 

„  _  ,i<x i  -  x>  <Yi  - 


‘  -  *>2 


(30) 


■ 

1-1 


and  the  estimate  b0  for  B0  can  be  obtained  from  the 
following. 


b0  -  Y  -  b2X 


(31) 


With  this  another  way  to  write  the  prediction  equation  is 
evident. 


Yx  =  f  +  ^(Xi  -  X)  (32) 

To  make  the  notation  simpler  and  somewhat  easier  to 
understand  for  larger  systems  it  is  often  advantageous  to 
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write  the  equations  in  matrix  notation.  The  corresponding 
matrix  equation  for  the  above  analysis  is  given  by  equation 


33. 


Y  =  XB  +  e 


(33) 


Y  is  the  dependent  variable  vector  of  size  n  by  1.  X  is  the 
augmented  independent  variable  matrix  of  size  n  by  2.  B  is 
the  parameter  vector  of  size  1  by  2  containing  the  scaler 
terms  B0  and  B^ .  The  matrix  normal  equation  can  be  written 
as  follows. 


XTXb  =  XtY 


(34) 


The  least  squared  estimate  for  b  is  then  obtained  from  the 
following  relation. 


b  =  (XtX)"1XtY 


(35) 


As  stated  earlier  the  model  desired  here  is  an 
autoregressive  source.  The  autoregressive  source  places  a 
further  restriction  on  the  regressive  model  in  that  the 
prediction  is  made  with  respect  to  a  time  series.  That  is 
future  values  of  a  time  series  are  to  be  predicted  on 
information  provided  by  past  values.  The  usual 
autoregressive  source  can  be  written  as 


xi+l  =  B0  +  Blxi  +  ei+l 


or  in  matrix  notation, 


X  *  ZB  +  e 


(36) 


(37) 
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Where  Z  represents  a  shifted  version  of  X.  In  the  particular 
case  at  hand,  the  derivation  has  to  be  carried  one  step 
further,  it  also  has  to  take  into  account  that  the  data  is 
two  dimensional.  The  two  dimensional  autoregressive  model 

as  stated  earlier  can  be  written  as, 

1  1 

I  (m, n, t)  =VVa(i,  j)I(m-dx-i,n-dy-j,t-r)  +  a0  +  e(i,j) 
i-0J=0  (38) 


Notice  that  this  is  not  in  the  ordinary  matrix  form.  Some 
manipulation  of  the  data  is  required  in  order  to  get  the 
system  into  a  form  that  is  readily  solvable  by  ordinary 
regression  analysis.  The  method  for  transforming  the  two 
dimensional  system  into  a  single  dimensional  regression 
problem  is  as  follows. 

1.  Rewrite  the  regression  coefficient  matrix  a(i,j)  into 
a  regression  vector  B  of  length  5  where  the  first 
value  comes  from  the  intercept  coefficient  a0 . 


B  = 


a0 

a(0,0) 

a(0,l) 

a(l,0) 

a(l,l) 


(39) 


2.  Set  up  the  independent  variable  matrix  Z  in  such  a 
way  so  as  to  take  into  account  the  shifting  that  is 
accomplished  through  the  use  of  the  double  summation. 
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In  the  equation  for  the  2  matrix  that  follows,  the  t- 
factor  has  been  neglected  for  simplicity  sake. 

1  I (i, j)  I(i, j+1)  I ( i+1 , j)  I(i+l,j+l) 

1  I(i, j+1)  I ( i , j+2)  I (i+1, j+1)  I ( i+1 , j+2) 

•  •  •  f 

•  •  •  • 

1  I ( i , j+J)  I ( i , j+J+1 )  I ( i+1 , j+J)  I ( i+1 , j+J+1)  (4 

1  I (i+1 , j)  I ( i+1 , j+1)  I (i+2 ,  j)  I (i+2, j+1) 

•  •  •  • 

•  •  •  t 

1  I ( i+I , j+J)  I(i+I, j+J+1)  I (i+I+1 , j+J)  I (i+I+1, j+J+1) 


Figure  16  shows  how  the  scanning  of  the  previous  data 
matrix  is  performed.  Note  that  each  value  is  used  up 


3.  Finally  row  scan  the  current  frame  block  data  placing 
it  in  a  column  vector  X  of  length  I  times  J,  where  I 
and  J  are  the  x  and  y  blocksize  measurements  of  the 
comparison  block.  The  scanning  method  is  graphically 
shown  in  figure  17.  Note  also  that  the  e  vector 
would  be  defined  identically  to  that  of  the  X  vector. 


I(ir jrt) 

I(i, j+l,t) 
I(i, j+2,t) 


X 


I(ifJrt) 
I(i+lr jft) 


(41) 


I(i+I, j+J,t) 

- 


With  each  of  the  variables  X,z,B,  ane  e  defined  the 
current  problem  reverts  to  that  of  a  first  order  regression 
problem  involving  five  regression  coefficients  and  hence  can 
be  solved  as  such.  The  predictor  coefficients  can  hence  be 
obtained  by  solving  the  following  matrix  equation  for  B. 

X  =  ZB  +  e  (42) 

The  normal  equation  is  then, 

ZTZB  =  ZTX  (43) 

and  the  least  squares  estimate  for  B  is  b  and  is  given  by 

b  -  (ZTZ)-1ZTX.  (44) 

As  with  any  system  that  is  based  on  obtaining  a  matrix 
inverse  it  is  possible  that  the  system  is  ill  conditioned 
and  the  inverse  may  not  exist.  In  the  current  system  this 
is  remidied  by  using  only  the  estimate  for  the  integer 
portion  of  the  displacement  if  the  current  status  proves  the 
system  to  be  ill  conditioned.  When  this  occurs  the  b  vector 
is  set  to  an  identity  transfer  function,  that  is, 

bT  =  [0,1,0,0,0]  (45) 

Solution  of  the  regression  problem  will  generate  the  four 
prediction  coefficients  and  the  intercept.  For  the  pure 
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translation-noiseless  case  there  is  a  very  nice  relationship 
amoung  these  coefficients.  First,  since  the  power  in  the 
past  and  present  frames  are  nearly  equal  the  coefficients 
will  sum  very  close  to  one.  Next  since  the  coefficients  act 
as  a  shifting  mechanism  each  will  be  greater  than  zero. 
Finally  because  of  the  symmetry  involved  in  the  values  of 
the  four  coefficient  they  can  be  thought  of  as  being  co¬ 
planer  and  the  off  sample  point  location  of  the  delta 
function  can  be  determined  by  finding  the  center  of  mass  of 
the  coefficients.  Hence  only  the  location  of  the  delta 
function  needs  to  be  transmitted  and  the  coefficients  can  be 
generated  from  this  knowledge.  When  the  motion  is  not  pure 
translation  and/or  noise  and/or  non-zero  background  is 
present,  then  it  advantageous  to  transmitt  the  values  of  the 
coefficients  instead  of  the  location  of  the  delta  function. 
With  noise,  rotation,  scale  change  and/or  other  non-optimal 
occurances  the  nice  features  listed  above  may  no  longer 
hold.  This  is  the  reason  for  the  transmission  of  the 
coefficient  values  themselves  and  not  the  delta  function 
location.  Hence  for  the  data  compression  scheme,  the 
quantized  prediction  coefficients  and  quantized  prediction 
errors  are  transmitted.  If  the  displacement  is  constant, 
then  the  coefficients  need  be  transmitted  only  for  the 
blocks  where  non-uniform  displacement  has  occured.  A  block 
diagram  if  the  system  is  given  in  figure  18  to  show  the 
order  in  which  each  of  the  steps  is  performed. 
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Figure  18.  System  Block  Diagram 
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QUANTIZED  ERROR  -  SECONDARY  BLOCKS I ZE 


CONCLUSIONS  AND  RESULTS 


With  the  assignment  of  a  time  modified  autoregressive 
model  for  interframe  image  coding  it  has  been  shown  that 
major  simplifications  can  be  made  by  taking  into  account 
nearest  integer  displacement.  The  limits  on  the  summations 
for  the  regression  no  longer  need  be  -p  to  p  to  resolve  for 
a  displacement  of  p  pixel  shift  in  either  direction,  but 
instead  can  be  limited  over  a  single  cell.  It  has  also  been 
shown  that  for  the  translation  only-noiseless  case  the 
prediction  coefficients  supply  enough  information  to 
determine  very  closely  the  actual  non-integer  displacement. 
When  combined  with  an  appropriate  coding  scheme  the  method 
should  produce  respectable  results  with  fewer  computations 
than  some  of  the  methods  previously  proposed.  It  offers  the 
advantages  of  not  having  to  go  through  an  iterative 
procedure  to  determine  the  displacement  as  well  as  being 
well  suited  for  parallel  implimentation.  Noise  in  the 
images  greatly  affects  the  quality  of  the  similarity  metric 
and  hence  generates  false  maximum  energy  concentration 
points.  The  problems  of  rotation,  scale  changes, 
perspective  changes  and  other  non-translation  types  of 
motion  can  only  be  resolved  with  the  addition  of  more 
intelligence  to  the  system. 

Three  sets  of  test  images  sequences  were  used  for  the 
algorithm  analysis.  The  first  is  a  sequence  of  frames  from 
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a  digital  television  transmission.  This  particular  sequence 
contains  every  other  frame  in  the  sequence  so  the  problem  of 
interlacing  that  the  original  data  possessed  could  be 
neglected.  The  sequence  consists  of  a  woman  talking  on  the 
telephone  while  at  the  same  time  making  some  rather  quick 
motions  with  her  head  and  hands.  This  particular  sequence 
will  be  referred  to  as  the  'FAST  PHONE'  in  the  remaining 
part  of  the  discussion.  The  second  image  sequence  was 
generated  identically  to  that  of  the  'FAST  PHONE'  sequence 
but  consisted  of  somewhat  slower  motion  in  the  sequence. 
The  final  sequence  consisted  of  an  in-house  motion  sequence 
generation.  A  fixed  background  was  placed  under  the  vidicon 
and  the  motion  was  achieved  by  moving  the  small  model  plane 
some  small  distance  between  the  image  frame.  These  last  two 
sequences  will  from  here  on  be  referred  to  as  the  'SLOW 
PHONE '  and ' PLANE ' . 

Figures  19  a  through  f  are  the  original  'FAST  PHONE' 
frames  while  figures  20  a  through  e  are  the  outputs  with  the 
associated  bit  rate  required  to  transmitt. 
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Figure  19.  "FAST  PHONE"  Input  Sequence. 


(a)  Input  Image  1.  (b)  Input  Image  2. 


(c)  Input  Image  3. 
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(d)  Input  Image  4. 
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Figure  19  Continued 


(c)  Output  Image  4  (.584) 


(d)  Output  Image  5  (1.00) 
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(g)  Output  Image  3  (.392) 


{,h)  Output  Image  4  (.  564) 


( 1)  Output  Image  5  (.893) 


(j )  Output  Image  6  (1.22) 


2 1  Con  t inued 


(e)  Input  Image  5.  (f)  Input  Image  6. 


Figure  22.  Output  Image  Sequence 


(a)  Output  Image  2  (.453)  (b)  Output  Image  3  (.693) 


0 


(a)  Input  Image  I 


(b)  Input  Image  2 


(c)  Input  Image  3 


( d )  Input  Image  4 


REFERENCES 


1.  Agin,  G.J.,  "Computer  Vision  Systems  for  Industrial 

Inspection  and  Assembly,"  IEEE  Computer,  Volume  13, 
Number  5,  May  1980,  pp.  11-20. 

2.  Anderson,  R.F.,  "TV-Compatible  Forward  Looking 

Infrared,"  Optical  Engineering,  Volume  13,  Number  4, 
July-August  1974,  pp.  335-338. 

3.  Anderson,  B.D.O.,  and  Moore,  J.B.,  Optimal  Filtering, 

Prentice-Hall  Inc.,  1979. 

4.  Andrews,  H.C.,  and  Hunt,  B.R.,  Digl.ta.1  Imags 

Restoration.  Prentice-Hall  Inc.,  1977. 

5.  Ayala,  I.L.,  Orton,  D.A. ,  Larson,  J.B.,  and  Elliot, 

D.F.,  "Moving  Target  Tracking  Using  Symbolic 
Registration,"  IEEE  Transactions  on  Pattern  Analysis 
and  Machine  Intelligence,  Volume  PAMI-4,  Number  5, 
September  1982,  pp.  515-520. 

6.  Barakat,  Richard,  "Application  of  the  Sampling  Theorem 

to  Optical  Diffraction  Theory,"  Journal  of  the 
Optical  Society  of  America,  Volume  54,  Number  7, 

July  1964,  pp.  920-930. 

7.  Barnes,  Casper  W.,  "Object  Restoration  in  a 

Diffraction-Limited  Imaging  System,"  Journal  of  the 
Optical  Society  of  America,  Volume  56,  Number  5,  May 
1966,  pp.  575-576. 

8.  Beall,  W.H.,  "Statistical  Analysis  of  Degradation  in 

Scanned  Image  Systems,"  Journal  of  the  Optical 
Society  of  America,  Volume  54,  Number  8,  August 
1964,  pp.  992-997. 

9.  Bode,  H.W.,  and  Shannon,  C.E.,  "A  Simplified  Derivation 

of  I  .'ear  Least  Squares  Smoothing  and  Prediction 
Theory,"  Proceedings  of  the  IRE,  Volume  38,  pp. 
417-425,  April  1950. 

10.  Bowling,  Carl  D. ,  and  Jones,  R.A. ,  "Displacement 

Estimation  by  Prediction  Coefficient  Energy 
Concentration,"  IEEE  Globall  Telecommunications 
Conference,  GLOBECOM'82,  Volume  1,  pp. 

B6.4.1-B6.4.4. 

-  B1  - 


11.  Budrikis,  Z.L.,  "Visual  Fidelity  Criterion  and 

Modeling,"  Proceedings  of  the  IEEE,  Volume  60, 

Number  7,  July  1972,  pp.  771-779. 

12.  Castleman,  K.R.,  Digital  Image  Processing .  Prentice- 

Hall  Inc.,  1979. 

13.  Chen,  Wen-Hsiung,  and  Smith,  C.  Harrision,  "Adaptive 

Coding  of  Monochrome  and  Color  Images,"  IEEE 
Transactions  on  Communications,  Volume  COM-25, 

Number  11,  November  1977,  pp.  1285-1292. 

14.  Cooper,  G.R.,  and  McGillem,  C.D.,  Probabilistic  Methods 

fiJL  Signal  and  System  Analysis .  Holt,  Rinehart,  and 
Winston  Inc.,  1971. 

15.  Danker,  A.J.,  and  Rosenfeld,  A.,  "Blob  Detection  by 

Relaxation,"  IEEE  Transactions  on  Pattern  Analysis 
and  Machine  Intelligence,  Volume  PAMI-3,  Number  1, 
January  1981,  pp.  79-92. 

16.  Daut,  D.G. ,  Fries,  R.W. ,  and  Modestino,  J.W. ,  "Two- 

Dimensional  DPCM  Image  Coding  Based  on  an  Assumed 
Stochastic  Image  Model,"  IEEE  Transactions  on 
Communications,  Volume  COM-29,  Number  9,  Sept‘.,;her 

1981,  pp.  1365-1374. 

17.  Dukhovich,  Isaac  J.,  and  O'Neal,  J.B.  Jr.,  "A  Three- 

Dimensional  Spatial  Non-Linear  Predictor  for 
Television,"  IEEE  Transactions  on  Communications, 
Volume  COM-26,  Number  5,  May  1978,  pp.  578-583. 

18.  Ferrie,  Frank  P. ,  Levine,  Martin  D. ,  and  Zucker,  Steven 

W. ,  "Cell  Tracking:  A  Modeling  and  Minimization 
Approach,"  IEEE  Transactions  on  Pattern  Analysis  and 
Machine  Intelligence,  Volume  PAMI-4,  Number  3,  May 

1982,  pp.  277-291. 

19.  Forgus,  R.H.,  and  Melamed,  L.E.,  Perception:  A 

Cognitive  Stage  Approach .  McGraw-Hill  Book  Company, 
1976. 

20.  Gallager,  R.G.,  Information  Theory  and  Reliable 

Communication.  John  Wiley  and  Sons  Inc.,  1968. 

21.  Gilbert,  Alton  L. ,  Giles,  Michael  K. ,  Flachs,  Gerald 

M. ,  Rogers,  Robert  B.,  and  U,  Yee  Hsun,  "A  Real-Time 
Video  Tracking  System,"  IEEE  Transactions  on  Pattern 
Analysis  and  Machine  Intelligence,  Volume  PAMI-2, 
Number  1,  January  1980,  pp.  47-56. 

22.  Gonzalez,  R.C.,  and  Wintz,  P.,  Digital  Image 

Processing .  Addison-Wesley  Publishing  Company  Inc. , 
1979. 

-  B2  - 


23.  Goyal ,  Shri  K.  ,  and  O'Neal,  J.B.  Jr.,  "Entropy  Coded 

Differential  Pulse-Code  Modulation  Systems  for 
Television,"  IEEE  Transactions  on  Communications, 
Volume  COM-23,  Number  6,  June  1975,  pp.  660-666. 

24.  Granrath,  Douglas  J. ,  "The  Role  of  Human  Vusual  Models 

in  Image  Processing,"  Proceedings  of  the  IEEE, 

Volume  69,  Number  5,  May  1981,  pp.  552-561. 

25.  Habibi,  Ali,  and  Wintz,  Paul  A.,  "Image  Coding  by 

Linear  Transformation  and  Block  Quantization,"  IEEE 
Transactions  on  Communications,  Volume  COM-19, 

Number  1,  February  1971,  pp.  50-62. 

26.  Habibi,  Ali,  "Hybrid  Coding  of  Pictorial  Data,"  IEEE 

Transactions  on  Communications,  Volume  COM-22, 

Number  5,  May  1974,  pp.  614-624. 

27.  Habibi,  Ali,  "Survey  of  Adaptive  Image  Coding 

Techniques,"  IEEE  Transactions  on  Communications, 
Volume  COM-25,  Number  11,  November  1977,  pp. 
1275-1284. 

28.  Habibi,  Ali,  "An  Adaptive  Strategy  for  Hybrid  Image 

Coding,"  IEEE  Transactions  on  Communications,  Volume 
COM-29,  Number  12,  December  1981,  pp.  1736-1740. 

29.  Harris,  J.L.,  "Diffraction  and  Resolving  Power," 

Journal  of  the  Optical  Society  of  America,  Volume 
54,  Number  7,  July  1964,  pp.  931-936. 

30.  Harris,  J.L.  Sr.,  "Image  Evaluation  and  Restoration," 

Journal  of  the  Optical  Society  of  America,  Volume 
56,  Number  5,  May  1966,  pp.  569-574. 

31.  Haskell,  Barry  G. ,  Gordon,  Pat  L. ,  Schmidt,  Robert  L. , 

and  Scattaglia,  James  V.,  "Interframe  Coding  of 
525-Line  Monochrome  Television  at  1.5  Mbits/s,"  IEEE 
Transactions  on  Communications,  Volume  COM-25, 

Number  11,  November  1977,  pp.  1339-1348. 

32.  Healy,  Donald  J. ,  Mitchell,  O.R.,  "Digital  Video 

Bandwidth  Compression  Using  Block  Truncation 
Coding,"  IEEE  Transactions  on  Communications,  Volume 
COM-29,  Number  12,  December  1981,  pp.  1809-1817. 

33.  Huang,  T.S.,  Picture  Processing  and  digital  Filtering, 

Spr inger-Verlag,  1975. 

34.  Jain,  R. ,  "Extraction  of  Motion  Information  from 

Peripheral  Processes,"  IEEE  Transactions  on  Pattern 
Analysis  and  Machine  Intelligence,  Volume  PAMI-3, 
Number  5,  September  1981,  pp.  489-503. 


B3 


35.  Jain,  Anil  K. ,  and  Angel,  Edward,  "Image  Restoration, 

Modelling,  and  Reduction  of  Dimensionality,"  IEEE 
Transactions  on  Computers,  Volume  C-23,  Number  5, 

May  1974,  pp.  470-476. 

36.  Jain,  Jaswant  R. ,  "Displacement  Measurement  and  Its 

Application  in  Interframe  Image  Coding,"  IEEE 
Transactions  on  Communications,  Volume  COM-29, 

Number  12,  December  1981  pp.  1799-1808. 

37.  Jain,  A.K.,  "Advances  in  Mathematical  Models  for  Image 

Processing,"  Proceedings  of  the  IEEE,  Volume  69, 
Number  5,  May  1981,  pp.  502-528. 

38.  Jain,  Ramesh,  and  Haynes,  Susan,  "Imprecision  in 

Computer  Vision,"  IEEE  Computer,  Volume  15,  Number 
8,  August  1982,  pp.  39-48. 

39.  Jarvis,  J.E.,  "Visual  Inspection  Automation,"  IEEE 

Computer,  Volume  13,  Number  5,  May  1980,  pp.  32-38. 

40.  Jarvis,  R.A.,  "A  Computer  Vision  and  Robotics 

Laboratory,"  IEEE  Computer,  Volume  15,  Number  6, 

June  1982,  pp.  8-22. 

41.  Jones,  R.A. ,  "Adaptive  Hybrid  Picture  Coding,"  SP IE, 

Vol.87,  Advances  in  Images  Processing  Techniques, 
1976,  pp.  247-255. 

42.  Jones,  R.A.,  "Adaptive  Hybrid  Picture  Coding," 

Technical  Proposal  D180-19071-1 ,  Boeing  Aerospace 
Company,  Research  &  Engineering  Division,  September 
1976. 

43.  Jones,  R.A. ,  "Adaptive  Hybrid  Picture  Coding,"  Interim 

Scientific  Report,  Department  of  Electrical 
Engineering,  University  of  Arkansas,  January  1979. 

44.  Jones,  R.A. ,  "Adaptive  Hybrid  Picture  Coding,"  Second 

Interim  Scientific  Report,  Department  of  Electrical 
Engineering,  University  of  Arkansas,  November  1979. 

45.  Kailath,  T. ,  "An  Innovations  Approach  to  Least-Squares 

Estimation.  Part  I:  Linear  Filtering  in  Additive 
White  Noise,"  IEEE  Transactions  on  Automatic 
Control,  Volume  AC-13,  Number  6,  December  1968,  pp. 
646-655. 

46.  Kailath,  T.  and  Frost,  P. ,  "An  Innovations  Approach  to 

Least-Squares  Estimation.  Part  II:  Linear  Smoothing 
in  Additive  White  Noise,"  IEEE  Transactions  on 
Automatic  Control,  Volume  AC-13,  Number  6,  December 
1968,  pp.  655-660. 


B4 


47.  Kogo,  Toshio,  Iijima,  Y.  ,  and  Iinuma,  K.,  "Statistical 

Performance  Analysis  of  an  Interframe  Encoder  for 
Broadcast  Television  signals,"  IEEE  Transactions  on 
Communications,  Volume  COM-29,  Number  12,  December 

1981,  pp.  1868-1876. 

48.  Kuo,  B.C.,  Digital  Control  Systems.  Holt,  Rinehart  and 

Winston  Inc.,  New  York,  1980. 

49.  Legters,  George  R.  Jr.,  and  Young,  Tzay  Y. ,  "A 

Mathematical  Model  for  Computer  Image  Tracking," 

IEEE  Transactions  on  Pattern  Analysis  and  Machine 
Intelligence,  Volume  PAMI-4,  Number  6,  November 

1982,  pp.  583-594. 

50.  Levy,  Armand  Jacques,  "Wiener  Estimation  for  Inversion 

of  Linear  Operators  and  Super  resolution, " 

Proceedings  IEEE  Computer  Society  Conference  on 
Pattern  Recognition  and  Image  Processing,  1981,  pp. 
433-436. 

51.  Liew,  Chong  Kiew,  "Inequality  Constrained  Least-Squares 

Estimation,"  Journal  of  the  American  Statistical 
Association,  Volume  71,  Number  355,  September  1976, 
Theory  and  Methods  Section,  pp.  746-751. 

52.  Liew,  Chong  K. ,  and  Shim,  Jae  K. ,  "A  Computer  Program 

for  Inequality  Constrained  Least-Squares 
Estimation,"  Econometr ica,  Volume  46,  Number  1, 
January  1978,  pp.  237. 

53.  Marchand,  E.W. ,  "Derivation  of  the  Point  Spread 

Function  from  the  Line  Spread  Function,"  Journal  of 
the  Optical  Society  of  America,  Volume  54,  Number  7, 
July  1964,  pp.  915-919. 

54.  Marks,  Robert  J.  II,  "Gerchberg's  Extrapolation 

Algorithm  in  Two  Dimensions,"  Applied  Optics,  Volume 
20,  Number  10,  May  15,  1981,  pp.  1815-1820. 

55.  Marks,  Robert  J.  II,  and  Smith,  Michael  J. ,  "Closed- 

form  Object  Restoration  from  Limited  Spatial  and 
Spectral  Information,"  Optics  Letters,  Volume  6, 
Number  11,  November  1981,  pp.  522-524. 

56.  Maxemchuk,  N.F.,  and  Stuller  J.A.,  "An  Intraframe  Codec 

Based  Upon  Non  Stationary  Image  Model,"  Bell  System 
Technical  Journal,  Volume  58,  Number  6  Part  II, 
July-August  1979,  pp.  1395-1412. 


B5 


57.  Maxemchuk,  N.F.,  and  Stuller,  J.A.,  "Reduction  of 

Transmission  Error  Propagation  in  Adaptively 
Predicted,  DPCM  Encoded  Pictures,"  Bell  System 
Technical  Journal,  Volume  58,  Number  6  Part  II, 
July-Sugust  1979,  pp.  1413-1423. 

58.  Meiri,  A.  Zvi,  and  Yedilevich,  E. ,  "A  Pinned  Sine 

Transform  Image  Coder,"  IEEE  Transactions  on 
Communications,  Volume  COM-29,  Number  12,  December 
1981,  pp.  1728-1735. 

59.  Melsa,  James  L.,  and  Sage,  Andrew  P. ,  Estimation  Theory 

with  Applications  Communications  and  Control. 
McGraw-Hill  Book  Company,  1971. 

60.  Melsa,  J.L.,  and  Cohn,  D.L. ,  Decision  and  Estimation 

Theory.  McGraw-Hill  Book  Company,  1978. 

61.  Meyers,  W. ,  "Industry  Begins  to  Use  Visual  Pattern 

Recognition,"  IEEE  Computer,  Vol.13,  Number  5,  May 

1980,  pp.  21-31. 

62.  Mitchell,  Owen  R. ,  and  Tavatavai,  A.J.,  "Channel  Error 

Recovery  for  Transform  Image  Coding,"  IEEE 
Transactions  on  Communications,  Volume  COM-29, 

Number  12,  December  1981,  pp.  1754-1762. 

63.  Mix,  Dwight,  F.,  Random  Signal  Analysis,  Addison-wesley 

Publishing  Company,  1969. 

64.  Modestino,  James  W. ,  and  Bhaskaran,  V.,  "Robust  Two- 

Dimensional  Tree  Encoding  of  Images,"  IEEE 
Transactions  on  Communications,  Volume  COM-29, 

Number  12,  December  1981,  pp.  1786-1798. 

65.  Modestino,  James  W. ,  Daut,  D.G. ,  and  Vickers,  A.L., 

"Combined  Source-Channel  Coding  of  Images  Using 
Block  Cosine  Transform,"  IEEE  Transactions  on 
Communications,  Volume  COM-29,  Number  9,  September 

1981,  pp.  1261-1274. 

66.  Modestino,  James  W.,  Bhaskaran,  V.,  and  Anderson,  J.B., 

"Tree  Encoding  of  Images  in  the  Presence  of  Channel 
Errors,",  IEEE  Transactions  on  Information  Theory, 
Volume  IT-27,  Number  6,  November  1981,  pp.  677-697. 

67.  Mohanty,  N.C.,  "Computer  Tracking  of  Moving  Point 

Targets  in  Space,"  IEEE  Transactions  on  Pattern 
Analysis  and  Machine  Intelligence,  Volume  Pami-3, 
Number  5,  September  1981,  pp.  606-611. 


B6 


68.  Mohwinkel,  C.,  and  Kurz,  L. ,  "Computer  Picture 

Processing  and  Enhancement  by  Localized  Operations," 
Computer  Graphics  and  Image  Processing,  Volume  5, 

1976,  pp.  401-424. 

69.  Natarajan,  T.  Raj,  Ahmed,  Nasir,  "On  Interframe 

Transform  Coding,"  IEEE  Transactions  on 
Communications,  Volume  COM-25,  Number  11,  November 

1977,  pp.  1323-1329. 

70.  Netravali,  A.N.,  and  Robbins,  J.D.,  "Motion-Compensated 

Television  Coding:  Part  I,"  The  Bell  System 
Technical  Journal,  Volume  58,  Number  3,  March  1979, 
pp.  631-669. 

71.  Netravali,  A.N.,  and  Stuller,  J.A.,  "Motion-Compensated 

Transform  Coding,"  Bell  System  Technical  Journal, 
Volume  58,  Number  7,  September  1979,  pp.  1703-1718. 

72.  Netravali,  Arun  N. ,  and  Limb,  John  0.,  "Picture  Coding: 

A  Review,"  Proceedings  of  the  IEEE,  Volume  68, 

Number  3,  March  198G,  pp.  366-406. 

73.  Netravali,  A.N.,  and  Bowen,  E.G.,  Bell  System  Technical 

Journal,  Volume  61,  Number  6,  Julv-August  1982,  pp. 
969-979. 

74.  Ngan,  K.N.,  "Adaptive  Transform  Coding  of  Video 

Signals,"  IEE  Proceedings,  Volume  329,  Part  F, 

Number  1,  February  1982,  pp.  28-40. 

75.  O'Brien,  Kathryn  B.,  "Automatic  Optical  Design  of 

Desired  Image  Distributions  Using  Orthogonal 
Constraints,"  Journal  of  the  Optical  Society  of 
America,  Volume  54,  Number  10,  October  1964,  pp. 
1252-1255. 

76.  Oppenheim,  Alan  V.,  Schafer,  Ronald  W.,  and  Stockham, 

Thomas  G. ,  "Nonlinear  Filtering  of  Multiplied  and 
Convolved  signals,"  Proceedings  of  the  IEEE,  Volume 
56,  Number  8,  August  1968,  pp.  1264-1291. 

77.  O'Rourke,  J. ,  "Motion  Detection  Using  Hough 

Techniques,"  IEEE  Computer  Society  Conference  on 
Pattern  Recognition  and  Image  Processing,  1981,  pp. 
82-87. 

78.  Ostrem,  J.S.,  and  Falconer,  D.G.,  "A  Differential 

Operator  Technique  for  Restoring  Degraded  Signals 
and  Images,"  IEEE  Transactions  on  Pattern  Analysis 
and  Machine  Intelligence,  Volume  PAMI-3,  Number  3, 
May  1981,  pp.  278-284. 


-  B7  - 


79.  Pack,  C.D. ,  and  Whitaker,  B.A.,  "Kalman  Filter  Models 

for  Network  Forecasting,"  Bell  System  Technical 
Journal,  Volume  61,  Number  1,  January  1982,  pp. 

1-15. 

80.  Pal,  S.K.,  and  King,  R.A.,  "Image  Enhancement  Using 

Smoothing  with  Fuzzy  Sets,"  IEEE  Transactions  on 
Systems,  Man,  and  Cybernetics,  Volume  SMC-11,  Number 
7,  July  1981,  pp.  494-501. 

81.  Papoulis,  a.,  Probability.  Random  Variables,  and 

Stochastic  Processes.  McGraw-Hill  Book  Company, 

1965. 

82.  Peacock,  K.L.,  and  Treitel,  Sven,  "Predictive 

Deconvolution:  Theory  and  Practice,"  Geophysics, 
Volume  34,  Number  2,  April  1969,  pp.  155-169. 

83.  Peleg,  Shmuel,  "A  New  Probabilistic  Relaxation  Scheme," 

IEEE  Transactions  on  Pattern  Analysis  and  Machine 
Intelligence,  Volume  PAMI-2,  Number  4,  July  1980, 
pp.  362-369. 

84.  Pirsh,  P.,  "Adaptive  Intra-Interframe  DPCM  Coder,"  Bell 

System  Technical  Journal,  Volume  61,  Number  5,  May- 
June  1982,  pp.  747-764. 

85.  Porter,  G.B.  and  Mundy,  J.L.,  "Visual  Inspection  System 

Design,"  IEEE  Computer,  Volume  13,  Number  5,  May 
1980,  pp.  40-48. 

86.  Pratt,  W.K.,  Digital  Image  Processing.  Wiley 

Interscience,  1978. 

87.  Price,  C. ,  Snyder,  W. ,  and  Rajala,  S. ,  "Computer 

Tracking  of  Moving  Objects  Using  a  Fourier-Domain 
Filter  Based  on  a  Model  of  the  Human  Visual  System," 
IEEE  Computer  Society  Conference  on  Pattern 
Recognition  and  Image  Processing,  1981,  pp.  98-102. 

88.  Rashid,  H.U.,  and  Jones,  R.A.,  "An  Adaptive  Estimation 

Approach  to  Estimation  Displacement  in  Time  Varying 
Images,"  SPIE  23rd  Internation  Symposium,  August 
1979,  San  Diego  California. 

89.  Rashid,  H.U.,  "An  Innovations  Approach  to  Displacement 

Estimation  and  Image  Analysis  in  Time  Varying 
Images,"  PHD  Dissertation,  Department  of 
Electrical  Engineering,  University  of  Arkansas, 
August  1980. 


-  B8  - 


**  *■ 


90.  Robbins,  John  D. ,  and  Netravali,  Arun  N. ,  "Interframe 

Telivision  Coding  Using  Movement  Compensation," 
Internation  Conference  on  Communications  Record, 
ICC'79,  Volume  2,  pp. 23 . 4 .1-23 .4 . 5 . 

91.  Robbins,  J.D.,  and  Netravali,  A.N.,  "Spatial 

Subsampling  in  Motion-Compensated  Television 
Coders,"  Bell  System  Technical  Journal,  Volume  61, 
Number  8,  October  1982,  pp.  1895-1910. 

92.  Roese,  John  A.,  Pratt,  William  K. ,  and  Robinson,  Guner 

S.,  "Interframe  Cosine  Transform  Image  Coding,"  IEEE 
Transactions  on  Communications,  Volume  COM-25, 

Number  11,  November  1977,  pp.  1329-1339. 

93.  Rosenf eld,  A.,  Picture  Processing  by  Computer.  Academic 

Press  Inc.,  1969. 

94.  Rosenfeld,  Azriel,  and  Kak,  Avinash,  Digital  Picture 

Processing.  Academic  Press  Inc.,  1976. 

95.  Sabri,  M.  Shaker,  and  Steenaart,  Willem,  "An  Approach 

to  Band-limited  Signal  Extrapolation:  The 
Extrapolation  Matrix,"  IEEE  Transactions  on  Circuits 
and  Systems,  Volume  25,  Number  2,  February  1978,  pp. 
74-78. 

96.  Sabri,  Shaker  "Movement-Compensated  Interframe 

Prediction  for  NTSC  Colour  TV  Signals,"  Paper 
Presented  at  the  Nato  Advanced  Study  Institute  on 
Image  Sequence  Processing  and  Dynamic  Scene 
Analysis,  Braunlage,  West  Germany,  June  1982. 

97.  Sakai,  Hajime,  and  Vanasse,  George  A,  "Direct 

Determination  of  the  Transfer  Function  of  an 
Infrared  Spectrometer,"  Journal  of  the  Optical 
Society  of  America,  Volume  56,  Number  3,  March  1966, 
pp.  357-362. 

98.  Schalkoff,  R.J.,  and  McVey,  E.S.,  "A  Model  and  Tracking 

Algorithm  for  a  Class  of  Video  Targets,"  IEEE 
Transactions  on  Pattern  Analysis  and  Machine 
Intelligence,  Volume  PAMI-4,  Number  1,  January  1982, 

pp.  2-10. 

99.  Shapiro,  Linda  G. ,  and  Haralick,  Robert  M., 

"Organization  of  Relational  Models  for  Scene 
Analysis,"  IEEE  Transactions  on  Pattern  Analysis  and 
Machine  Intelligence,  Volume  PAMI-4,  Number  6, 
November  1982,  pp.  595-602. 


100.  Smith,  David  K.,  and  Marks,  Robert  J.  II,  "Closed  Form 
Bandlimited  Image  Extrapolation,”  Applied  Optics, 
Volume  20,  Number  14,  July  15  1981,  pp.  2476-2483. 


101. 

102. 

103. 

104. 

105. 

106. 

107. 

108. 

109. 

110. 

111. 


Stark  H. ,  Cahana,  D. ,  and  Webb,  H. ,  "Restoration  of 

Arbitrary  Finite-Energy  Optical  Objects  from  Limited 
Spatial  and  Spectral  Information,"  Optical  Society 
of  America,  Volume  71,  Number  6,  June  1981,  pp. 
635-642. 

Stockham,  Thomas  G.,  "Image  Processing  in  the  Context 
of  a  Visual  Model,"  Proceedings  of  the  IEEE,  Volume 
60,  Number  7,  July  1972,  pp.  828-842. 

Stuller,  J.A.,  Netravali,  A.N.,  and  Robbins,  J.D., 
"Interframe  Television  Coding  Using  Gain  and 
Displacement  Compensation,"  Bell  System  Technical 
Journal,  Volume  59,  Number  7,  September  1980,  pp. 
1227-1240. 

Tou,  J.T.,  and  Gonzalez,  R.C.,  Pattern  Recognition 
Principles.  Addison-Wesley  Publishing  Company, 
Reading,  Mass.,  1974. 

Webb,  J.A. ,  and  Aggarwal,  J.K.,  "Visual  Interpretation 
of  the  Motion  of  Objects  in  Space,"  IEEE  Computer 
Society  Conference  on  Pattern  Recognition  and  Image 
Processing,  1981,  pp.  516-521. 

Williams,  R. A. ,  and  Chang,  W.S.C.,  "Resolution  and 

Noise  in  Fourier-Transf orm  Spectroscopy,"  Journal  of 
the  Optical  Society  of  America,  Volume  56,  Number  2, 
February  1966,  pp.  167-170. 

Winston,  Patrick  Henry,  The  Psychology  of.  Computer 
Vision.  McGraw-Hill  Book  Company,  1975. 

Wintz,  Paul  A.,  "Transform  Picture  Coding,"  Proceedings 
of  the  IEEE,  Volume  60,  Number  7,  July  1972,  pp. 
809-820. 

Youla,  Dante  C. ,  "Generalized  Image  Restoration  by  the 
Method  of  Alternating  Orthogonal  Projections,"  IEEE 
Transactions  on  Circuits  and  Systems,  Volume  25, 
Number  9,  September  1978,  pp.  694-702. 

Zadeh,  L.A.,  and  Ragazzini,  "An  Extension  of  Wiener's 
Theory  of  Prediction,"  Journal  of  Applied  Physics, 
Volume  21,  pp.  545-655,  July  1950. 

Zschunke,  Willmut,  "DPCM  Picture  Coding  with  Adaptive 
Prediction,"  IEEE  Transactions  on  Communications, 
Volume  COM-25,  Number  11,  November  1977,  pp. 
1295-1302. 


B10 


PART  TWO 


MACHINE  RECOGNITION  OF  PARTIAL  SHAPES 
USING 

FEATURE  VECTORS  DEFINED  ON  SHAPE  SPACE 


V 


MACHINE  RECOGNITION  OF  PARTIAL  SHAPES 
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FEATURE  VECTORS  DEFINED  ON  SHAPE  SPACE 


INTRODUCTION 

In  this  paper  the  problem  of  determining  whether  a 
partial  shape  belongs  to  a  more  complex  whole  shape  is 
addressed.  A  partial  shape  occurs  when  a  shape  is  partially 
obscured  by  another  shape,  or  object,  or  is  partly  in  the 
field  of  view. 

At  present  the  methods  of  recognizing  shapes  [5],  [15], 
[21],  [22],  can  be  categorized  as  either  global  or  local  in 
nature.  Within  the  class  of  global  shape  analysis  algorithms 
there  are  two  categories  that  under  certain  circumstances 
possess  the  ablity  to  recognize  complete  or  whole  shapes 
independent  of  size,  rotation,  or  location.  These  are  the 
Fourier  descriptors  methods  and  the  Syntactic,  or  Graphical, 
methods.  The  Fourier  descriptors  based  algorithm  perform 
satisfactorily  on  complete  shapes.  The  Fourier  coefficients 
extracted  are  indeed  independent  of  size,  rotation,  and 
location  when  the  shapes  are  complete.  However  this  method 
does  not  perform  satisfactorily  and  in  fact  fails  entirely 
when  the  class  of  shapes  is  allowed  to  include  incomplete  or 
partial  shapes.  An  example  is  presented  in  Section  II  that 
demonstrates  that  Fourier  descriptors  method  fails  to  work 
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on  incomplete  shapes.  The  results  of  the  experiment  are 
discussed  in  order  to  point  out  specifically  why  the 
algorithm  cannot  perform  satisfactorily  on  partial  shapes. 

The  other  class  of  global  methods,  namely  the  Syntactic 
methods  [4]  have  restricted  use  in  recognition  of  shapes 
because  these  algorithms  tacitly  assume  a  priori  that  the 
shapes  have  been  identified  by  their  parts.  These  algorithms 
then  investigate  the  relationship  between  the  various  parts 
of  the  shape.  A  human  shape,  for  instance  has  a  hand  or  a 
face  at  some  definite  orientaion  and  location  with  respect 
to  each  other. 

The  Local  category  of  shape  analysis  algorithm  [21]  uses 
'curvature'  as  a  criterion  for  detecting  the  peaks  and 
valleys  of  a  shape.  These  peaks  and  valleys  are  called  the 
local  shape  descriptors.  This  shape  comparison  algorithm  is 
not  independent  of  rotation.  In  Section  II  specific  examples 
are  given  that  demonstrate  that  the  present  comparison  type 
local  shape  analysis  algorithms  are  not  independent  of 
rotation.  The  concept  of  curvature  is  then  presented  from 
the  point  of  view  of  differential  geometry  for  determining 
why  the  local  shape  descriptors  are  not  independent  of 
rotation. 

In  Section  III,  several  general  concepts  from  the 
allometric  disciplines  [11] ,  are  combined  with  some  entirely 
new  concepts  concerning  shapes  for  the  purpose  of  providing 
the  foundation  for  a  new  approach  to  defining  shape  as 
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vectors  in  the  appropriate  space.  The  properties  of  this 
space  are  stated  definitively,  after  the  concept  of  size 
variable  has  been  solidified.  This  vector  space  is  called 
shape  space.  Two  theorems  useful  in  the  partial  shape 
recognition  problem  are  stated  and  proved  utilizing  shape 
space  properties. 

In  Section  IV,  some  basic  definitions  regarding  critical 
points  are  presented.  These  definitions  are  used  in  Section 
V  where  one  of  the  principal  results  of  this  paper  is 
presented.  Specifically,  a  new  procedure  of  determining  the 
critical  points  of  a  shape  is  described.  This  procedure  is 
named  the  Adaptive  Line  of  Sight  method.  In  the  Adaptive 
Line  of  Sight  method,  the  critical  point  determination  in 
based  on  a  set  of  coordinate  axes  that  are  dependent  on  the 
shape  being  examined.  Examples  are  given  that  demonstrate 
that  the  procedure  produces  critical  points  that  are 
independent  of  rotaion,  size,  displacement,  and  correspond 
closely  to  those  produced  by  normal  human  cognitive  process. 

It  is  demonstrated  in  Section  VI  that  measurements 
between  a  set  of  critical  points  that  are  determined  by 
using  the  Adaptive  Line  of  Sight  method  can  be  used  to 
define  feature  vectors  for  shapes.  These  feature  vectors 
remain  the  same  whether  the  shape  is  a  partial  shape  or  a 
more  complex  whole  shape.  Moreover,  these  feature  vectors 
are  independent  of  size,  rotation  and  displacement  since 
they  are  derived  from  a  set  of  critcal  points  that  are 
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independent  of  the  same  quantities.  A  procedure  for 
comparing  the  feature  vectors  of  a  set  of  shapes  is 
described  The  comparison  procedure  is  based  on  a  Syntactic 
method  which  will  point  out  whether  one  shape  is  part  of  a 
more  complex  whole  shape,  or  whether  the  shapes  are  totally 
dissimilar . 
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SECTION  II 


PRELIMINARY  THEORY  AND  DEFINITIONS 


There  exist  two  types  of  Fourier  descriptors.  The  first 
type  of  descriptors,  used  by  Zahn  and  Roskies  [22] ,  have 
been  called  descriptors  Sn  by  Pavlidis  [13],  In  the  method 
of  descriptors  Sn  the  shape  is  represented  by  the  continu¬ 
ous  function. 


where 


a (  t (k) ,k)  -♦<  k  )  +  t(  k  )  (2.1) 

t(  k  )  =  2ffp(k)/L 

p (  k  )  =  arc  length  between  the  starting 

point  and  the  k  th  point  on  the 
curve. 

<M  k  )  =  net  amount  of  angular  change 

between  the  starting  point  and 
the  k  th  point  on  the  curve. 

L  =  the  perimeter  of  the  curve 


The  descriptors  Sn  for  a  continuous  shape 
as. 


k) exp ( —  j  2 wnt) dt 


are  then  defined 


(2.2) 


These  descriptors  exhibit  some  notable  shortcomings.  Among 
these  are  1)  the  property  of  the  closure  of  the  curve  is  not 
preserved  2)  simple  shapes  such  as  squares  and  triangles 
cannot  be  distinguished  from  one  another  when  only  the  ver- 
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tices  are  given.  The  second  type  of  Fourier  descriptors, 
namely  the  descriptors  T^[13],  [15],  [17]  exhibit  character- 
sitics  that  are  superior  to  the  descriptors  Sn  in  the 
sense  that  the  reconstruction  of  the  shape  from  a  finite  set 
of  coefficients  leads  to  a  closed  curve.  Also  the  conver¬ 
gence  properties  are  superior. 

For  the  descriptors  Tn  the  shape  data  is  represented  in 
the  complex  form, 

u(p)  =  x(p)  +  j  y (p )  (2.3) 

where  (x(p)  ,  y(p))  are  the  coordinates  of  the  point  on  the 

curve  and  p  is  the  arc  length  from  the  defined  starting 

point.  The  descriptors  Tn  are  then  given  by, 

L 

T  =1  I  u(  p  )  exp  {- j2ivnp/L)  dl  (2.4) 

L  J 

0 

The  Fourier  descriptors  based  algorithm  normalizes  for 
position  by  setting  Tq  to  zero  .  Normalization  for  scale, 
rotation,  and  starting  point  of  a  contour  is  achieved  by 
multiplying  the  n  th  coefficient  by  s  exp(  j  ( $  +na  )).  The 
parameter  s  scales  the  shape  to  the  normalized  size,  the  pa¬ 
rameter  ( $  +  na  )  rotates  the  contour  to  the  normalized 
position.  The  normalization  is  such  that  the  coefficients 
T+^  and  T_^  are  pure  imaginary  numbers  and  their  sum  has 
magnitude  one. 
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It  is  easy  to  see  why  such  a  normalization  is  necessary 
for  discrete  data.  In  this  case,  the  Fourier  descriptors  are 


given  by, 

N-l 

T(  n  )  =  ^  e  (  x(  k  )  +  j  y{  k  ))exp(  -j2irnk/N)  (2.5) 
k=0 

Multiplying  (2.5)  by  s  exp-j(4>  +  not  )  the  normalized  coeffi¬ 
cients  ,  Tn  (  n  )  are  then  given  as, 

Tn(  n  )  =  T(  n  )  s  exp  j  ( $  +  n<*  )  (2.6) 

Next  imposing  the  requirement  on  (2.6)  ,  that  the  coefficient 

be  purely  imaginary  at  n  =  +  1  leads  to  the  condition  that, 

2-rrk  -(<(>  +  a)  =  2m  +  (-  2  tt  k  -  (  <J>  -  a))  (2.7) 

N  N 

where  m  is  an  integer. 

Setting  the  real  part  to  zero  by  appropriate  choices  of <}>  and 
a  is  equavalent  to  normalizing  for  rotation  and  starting 
point  of  the  contour.  As  a  matter  of  fact,  the  two  equa¬ 
tions  could  have  been  set  equal  to  any  constant. 

The  imaginary  parts  of  the  normalized  coefficient  at  n  = 
+1  are, 

s  N'X 

IM(T„(  1  ) )  =  N  Z  (-x  (k)  sin  (2_nk  -  ( <t>  +cQ  )  +y  (k)  cos  (2  ”k  -  (<t>+o)) 
N  k=0  N  N 

s  N_1 

IM  (T.,(-l  ))  =  lj  z.  (-x  (k)  sin  (2Trk  -($  -a))  +  y(k)cos(2jrk  -(<j>-a)) 
N  N  k=0  ~W  N 

Summing  and  equating  the  magnitude,  of  the  normalized  coef¬ 
ficients  at  ±1,  to  one,  yields, 


In  a  real  shape  recognition  application  it  is  not  known  a 
priori  whether  the  shape  under  examination  is  a  part  of  a 
more  complex  shape  or  a  shape  in  its  own  right.  The  above 
descriptors  in  their  present  form  are  not  capable  of  recog¬ 
nizing  that  a  partial  shape  may  be  part  of  a  more  complex 
shape.  We  present  an  example  to  demonstrate  the  validity  of 
this  contention.  The  shapes  used  are  shown  in  Fig.  1.  Figure 
1-a  is  a  complete  shape,  namely  a  swept  wing  plane,  while 
Figure  1-b  is  a  partial  shape,  namely  the  front  part  of  the 
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FIG.  2.  FOURIER  DESCRIPTORS  CALCULATED  USING  THE 


totmiKN  unsown  oh 


FIG 


plane.  The  plots  of  the  real  and  imaginary  part  of  the  nor¬ 
malized  Fourier  descriptors  obtained  by  using  (2.8) ,  for  the 
complete  shape  are  shown  in  Fig.  2-a.  The  corresponding 
plots  for  the  partial  shape  are  are  shown  in  Fig.  2-b.  Sim¬ 
ilar  data  obtained  using  (2.9)  is  shown  in  Fig.  3.  It  is 
apparent  that  comparing  the  two  sets  of  data  yields  nothing 
more  that  a  statement  that  the  two  shapes  are  dissimilar. 
Two  explanations  for  this  are  1)  the  parameters  s  and  o  are 
not  independent  of  the  shape  and  2)  the  Fourier  descriptors 
method  compares  the  'frequencies'  of  the  shapes.  The  fre¬ 
quencies  of  the  partial  shape  are  not  the  same  as  the  fre¬ 
quencies  of  the  complete  shape. 

The  main  result  of  this  paper  is  the  description  of  an 
approach  for  determining  if  a  partial  shape  belongs  to  a 
more  complex  whole  shape.  This  is  detailed  in  section  III 
IV  and  V. 

II-B  LOCAL  SHAPE  DESCRIPTORS 

The  Local  Shape  descriptors  method  has  been  used  by  Wal¬ 
lace,  Mitchell  and  Fukunaga  [21],  for  comparing  shapes 
stored  in  a  library.  In  this  method,  the  angle  function  is 
defined  as, 

s (k) =arctan ( (y(k)-y(k-l) )/(x (k) -x(k-l) )  (2.10) 
Curvature  is  then  defined  as  the  derivative  of  the  angle 
function,  that  is  s'(  k  ).  The  peaks  and  valleys  in  s'(  k  ) 


CARDIOID  OF  SIZE = A 


Y 

I  25- 

100- 

75- 

50- 

25- 


T  . . . . . I . . . r'”" . . . . .  "  . . . i 

•  00  -7S  -S0  -25  0  2S  50  75  100  125 

X 


C-CRITItAL  POINTS  POUND  BY  TAKING  THE  DERIVATIVE  OP  THE  ANGLE 


CARDIOID  OF  SIZE = A  ROTATED  BY  1.57  RADIANS 


•  25 


C -CRITICAL  POINTS  POUND  BY  TAKING  THE  DERIVATIVE  OP  THE  ANGLE 
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are  used  for  finding  the  peaks  and  valleys  of  the  curve. 
Each  local  shape  descriptor  then  consists  of  two  adjacent 
peaks  and  a  valley  (  alternate  angles  and  a  distance) . 

Observe  that  (2.10)  is  a  nonlinear  transformation  on  the 
data,  also,  the  function  is  not  finite  at  +  90°  In  general, 
the  function  may  be  used  to  locate  peaks  where  1)  the  angle 
between  any  three  adjacent  points  is  acute  2)  the  shape  be¬ 
comes  parallel  to  the  y-axis.  This  is  equivalent  to  places 
where  the  angle  becomes  +  |-  radians  with  respect  to  the  x- 
axis.  To  illustrate  this  point  the  cardioid  shown  in  Fig. 
4-a  was  generated  sampling  at  constant  intervals  of 
(2  tv/256)  radians  and  not  the  arc  length,  so  that  an  acute 
angle  did  not  occur  between  any  three  adjacent  points.  The 
cardioid  was  then  rotated  by  1.57  radians.  The  corresponding 
plots  for  the  angle,  and  the  derivative  of  the  angle  func¬ 
tion,  and  the  peaks  obtained  using  this  method  are  shown  in 
Fig.  5  and  Fig.  6,  for  both  cases.  Note  peaks  obtained  by 
this  method  are  not  unique  and  are  dependent  on  the  rota¬ 
tion. 

Curvature  is  known  [7]  to  be  a  property  of  a  curve  inde¬ 
pendent  of  its  orientation  in  space.  Despite  the  fact  that 
this  orientation  independent  quantity  was  used  to  determine 
peaks,  the  peaks  were  not  independent  of  rotation.  Clarifi¬ 
cation  of  this  apparent  contradiction  requires  the  concepts 
of  curvature  and  torsion  of  curves  in  space. 


CURVES  IN  THREE  DIMENSIONAL  SPACE 

The  problem  of  shape  recognition  is  analogous  to  recognition 
of  curves  in  space.  Therefore,  well  known  concepts  and  theo¬ 
rems  from  differential  geometry  can  be  utilized  in  shape 
analysis.  A  theorem  from  differential  geometry  that  is  par¬ 
ticularly  useful  is  [10], 

Theorem:  Every  regular  curve  c:Ig  -*•  Rncan  be  parameterized 

by  arc  length. In  other  words,  given  a  regular  curve  c:Ig-  Rn 
there  is  a  change  of  variables  e  :Ig  I0  such  that 
j(c. 9) ' (s) | =1 ,  where  (c.e)is  a  composite  function. 


Let  c  =  c(  s  )  be  the  parametric  representation  of  the 
curve  under  analysis  with  s  as  the  natural  parameter (i .e 
|dc/ds  1  =  1)  then  the  vectors  t,  n,  b,  satisfy  the  Serret- 
Frenet  equations  [7] ,  [10] , 


(2.11) 


where , 

<  =  curvature, 
x  =  torsion, 
t  =  tangent, 

n  =  normal  at  the  point, 
b  =  binormal  at  the  point, 
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and  the  dot  notation  denotes  the  deravative  with  respect  to 
the  natural  parameter  s. 

When  the  curve  under  analysis  lies  in  a  plane,  the  torsion  z 
in  general  is  equal  to  zero  and  the  binormal  b  is  constant. 
Thus  for  a  curve  in  the  x-y  plane  (2.11)  reduces  to, 


~.  -1 
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Now  if  0  is  the  angle  made  with  respect  to  the  x-axis  by  the 
tangent  to  the  curve  then, 
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where  i  and  j  are  unit  vectors  in  the  x  and  y  directions  re¬ 
spectively,  Differentiating  (2.13)  with  respect  to  the  natu- 


ral  parameter  s 

yields. 
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Substituting  equation  (2.13) 

in 

result. 
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Comparing  (2.12) 

with  ( 

2.16) 

< 

e 

COS  9 


(2.14) 


(2.15) 
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(2.16) 


In  view  of  the  different  representations  of  the  word  cur¬ 
vature  [2],  [21]  it  is  necessary  to  emphasize  that  the  de¬ 

rivative  in  (2.16)  is  with  respect  to  the  natural  parameter 
s  and  not  with  respect  to  some  arbitrary  distance  measure. 
If  a  curve  is  not  represented  in  ‘-erms  of  the  natural  param¬ 
eter,  but  some  other  real  valued  function  (say  6 *e (  s  )  ) 

then  this  transformation  should  be  allowable.  The  implica¬ 
tion  is  that,  e  : Ig  -  I  , is  an  injective  mapping  of  the  in¬ 
terval  I  onto  I  .  Where  I  and  Ifi  are  the  respective  do- 
s  y  s 

mains  in  s  and  9  over  which  the  curve  is  defined. 


Thus  the  function  arctan(  6)  with  values  between  +  90  or 
arccos(  9)  with  9  between  0  and  180  are  not  allowable  chang¬ 
es  of  parameter  and  any  property  based  on  these  transforma¬ 
tions  may  not  be  a  property  of  the  curve  but  a  property  of 
the  representation.  Thus  if  a  curve  or  a  shape  has  been  rep¬ 
resented  in  terms  of  the  sample  number  ' k  *  and  if  the  algor¬ 
ithm  is  unable  to  affect  the  one-to-one  transformation  de¬ 
scribed  above,  then  the  following  fact  should  be  exploited 


[10]  , 

ds  dc 

dk  dk 


(2.17) 


Using  (2.17)  it  can  be  shown  [10]  that  the  magnitude  of  cur¬ 
vature  can  be  obtained  from  the  following  relationship, 


|C'  X  c 

0 


..  I  3 


if  c' ^  0 
otherwise 


(2.18) 
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Where  the  symbol  x  denotes  the  vector  cross-product  and  the 
symbol  1  denotes  differentiation  with  respect  to  k.  Curva¬ 
ture  is  a  vector  quantity  and  it  points  in  the  direction  of 
the  normal. 

Using  equation  (2.18)  on  the  cardioid  and  its  rotated 
version  (  Fig.  4a  and  4b  )  resulted  in  the  plots  of  the  cur¬ 
vature  magnitudes  shown  in  Fig.  6a  and  6b.  As  predicted  the 
magnitude  of  the  curvature  is  identical  before  and  after  ro¬ 
tation.  This  is  in  sharp  contrast  to  substantial  differences 
demonstrated  in  Fig.  5a  and  b. 

SECTION  III 

BASIC  SHAPE  CONCEPTS 

In  this  section  several  general  concepts  from  allometric 
disciplines  [11]  are  combined  with  some  entirely  new  con¬ 
cepts  concerning  shapes.  This  combination  provides  the  foun¬ 
dation  for  a  new  approach  to  defining  shape  vectors  in  the 
appropriate  shape  space.  This  new  space  is  a  combination  of 
properties  of  vector  spaces  and  is  called  definitively  shape 
space. 

Applying  the  shape  space  concepts  to  the  shape  analysis 
problem  provides  a  basis  for  the  recognition  that  the  fea¬ 
tures  of  two  or  more  shapes  under  analysis  are  the  same.  For 
instance,  with  shape  space  concepts  it  is  possible  to  deter- 


mine  that  a  partial  shape,  independent  of  size  or  rotation, 
belongs  to  a  more  complex  whole  shape. 


Typically  in  a  shape  analysis  problem,  an  algorithm  op¬ 
erates  on  the  shape  data  according  to  a  set  of  criteria  for 
the  purpose  of  reaching  a  decision  of  some  sort.  Usually  the 
decision  is  whether  or  not  two  or  more  shapes  are  the  same. 
The  shape  analysis  algorithm  utilizes  measurements  such  as 
curvature,  and  measurements  between  predefined  points  on  the 
shape,  such  as  length  width,  diameter,  area,  etc.  Therefore 


if 

K  shapes 

are  under  analysis. 

and  N 

measurements  are 

made 

on 

each 

shape,  then  the 

result 

is  the 

K  measurement 

vectors 

t 

M1 

= 

""u 

, m-^2  » m^3  >  •  • 

•  t 

mlN  ) 

• 

= 

• 

•  • 

• 

• 

Mk 

= 

"Yi 

'mk2  'mk3  '*  * 

•  t 

mkN  } 

(3.1) 

mk 

= 

(mKi 

,  mK2  ^3  f  •  • 

• 

'  "Vn  ) 

where  the  first  subscript  of  m^  refers  to  the  object  be¬ 
ing  measured  while  the  second  subscript  i  refers  to  the  i  th 
measurement  between  the  predefined  points  on  the  k  th  shape. 
Any  two  objects  will  then  be  said  to  have  the  same  shape 
with  respect  to  these  measurements  if  one  vector  is  a  scalar 
multiple  of  the  other, 

«k  ”  a  Mp 
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(3.2) 


FIG.  7.  EXAMPLES  OF  SIZE-VARIABLE  IN  2-D  SHAPE  SPACE 


where  a  is  a  scalar  greater  than  zero. 

The  geometric  significance  of  (3.2)  is  that  in  the  N  dimen¬ 
sional  space  of  positive  measurements  all  points  on  a 
straight  line  through  the  origin  define  the  same  shape. 
Points  can  be  uniquely  located  on  the  positively  directed 
line  by  finding  the  intersection  of  first  order  surfaces 
with  the  line  defining  the  shape.  The  class  of  functions 
which  define  these  surfaces  are  linear  homegenous  functions 
of  order  one,  of  the  measurements,  m  i=l,  2,.N.  The  mathe¬ 
matical  representating  for  this  class  is, 


H  (a  m^)  -  a  H 
5  (mki>  :  0 


(3.3) 


where  a  is  scalar 

and  H  refers  to  a  countably  infinite  class  of  linear 
functions,  with  members  Z^  (  )  • 

The  distance  from  the  origin  to  the  intersection  of  a 
particular  member  of  the  class  (  rr^  )  with  the  ray 
through  origin  defining  the  shape  is  refered  to  as  the  size, 
scale  factor  or  the  normalization  factor  with  respect  to 
that  member.  Following  Mosain.an  [11]  terminilogy,  in  the 
sequel  Zj  (  ra  ^  )  will  be  refered  to  as  the  size  variable. 
Some  examples  of  size  variable  in  a  measurement  space  of  two 
vectors  are  shown  in  Fig.  7a  and  b. 


-  67  - 


These  statements  merit  further  discussion  and  clarifica¬ 
tion  because  of  their  implications.  Consider  for  example  the 
two  different  measurement  vectors  extracted  from  the  simple 
shape  shown  in  Fig.  8.  The  shape  is  a  unit  square.  In  the 
first  case,  the  measurement  vector  is. 

Mi®  (  ,  m12  t  m^  ,  m^  )  =  (  1  ,1  ,1.414,  1)  (3.5) 

while  in  the  second  case,  the  measurement  vector  is 

M2'  =  (  m2i  ,m£2  ,  m23  ,  m^  )  =  (  1,  1,  1,  1)  (3.6) 

where  the  1  in  the  above  equation  indicates  that  the  meas¬ 
urements  are  between  a  different  set  of  feature  points  of 
the  shape.  Now  if, 

Zj  "ki  »  *  \3  <3‘7) 

is  chosen  as  the  size  variable  then  the  corresponding  shape 

vectors  are, 

Sx  =(.707  ,  .707  ,  1  ,  .707)  (3.8) 

S2  =(  1  ,  1,1,  1) 

Comparing  these  two  shapes  without  any  reference  to  the  size 
variable  or  the  points  between  which  the  measurements  were 
made,  one  would  conclude  that  the  the  two  shapes  are  not  the 
same.  Thus  not  only  has  the  functional  form  of  the  size 
variable  to  be  the  same  but  the  measurements  involved  in  the 
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functional  relationship  have  to  be  between  the  same  feature 
points.  Now  consider  the  shapes  in  Fig.  9.  This  is  the  typ¬ 
ical  situation  in  which  occlusion  occurs  or  the  shape  is 
outside  the  field  of  view  of  a  camera  (or  some  other  measur¬ 
ing  device) .  Assume  that  both  shape  boundaries  are  repre¬ 
sented  by  an  equal  number  of  samples  and  that  every  point  on 
each  shape  boundary  is  defined  as  a  feature  point.  Now  if 
the  standard  deviation  of  the  data  is  chosen  as  the  size 
variable  then  it  is  obvious  that  the  standard  deviation  of 
the  two  shape  boundaries  are  different.  Therefore,  this  is  a 
case  where  the  size  variable  is  dependent  on  the  shape, 
which  implies  that  the  shape  vector  is  dependent  on  the  size 
variable.  Comparision  of  the  two  shape  vectors  with  such  a 
size  variable  is  bound  to  lead  to  errors.  These  problems 
can  be  alleviated  by  defining  the  quantities  in  the  proper 
context.  This  can  be  accomplished  only  if  the  variables  are 
defined  in  the  proper  space. 

SHAPE  SPACE 

In  this  section,  the  concept  of  a  sriape  space  is  intro¬ 
duced.  The  space  is  defined  in  terms  of  its  properties  in 
the  usual  manner  and  then  two  theorems  addressing  the  prob¬ 
lem  of  partial  shapes  are  stated  and  proved.  Assuming  that 
SRj(ski)  a  shape  vector  consisting  of  well  defined  oper¬ 
ations  on  the  measurements  of  the  shape  under  consideration. 


It  will  be  shown  in  the  sequel  that  it  is  necessary  to 
choose  a  size  variable  that  is  independent  of  the  shape  un¬ 
der  comparison.  A  necessary  first  step  is  the  definition  of 
shape  vector  [11]. 

SHAPE  VECTOR 


The  Shape  vector  S^jis  defined  to  be  the  ratio  of  the 


measurement  vector  M, 


to  the  size  variable  Z . . 


sy  ■ 


Vzj 


skj=(  mkl 


m 


k2 


m 


k3 


ZTTi^Ty  z. (mki)  Z. (mki) 


'  \n  } 


(3.4) 


It  may  be  noted  that  the  first  subscript  of  corresponds 
to  the  object  whose  shape  is  under  consideration  while  the 
second  subscript  corresponds  to  the  size  variable  chosen 
from  the  class. 


Here  the  points  to  be  emphasized  are  that, 

1 )  All  measurements  are  made  between  pre-defined  points. 

2) A  shape  vector  is  defined  with  respect  to  a  size  variable 
Zj  (  m^).  Thus  only  shape  vectors  defined  with  respect  to 
the  same  size  variable  can  be  compared. 

3)  If  two  shape  vectors  are  equal,  then  the  two  objects  have 
the  same  shape  with  respect  to  the  measurements. 

4)  The  shape  vector  should  be  independent  of  the  size 
variable. 
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These  measurements  are  the  m  's  previously  defined;  the 
elements  of  S .  ( 3ic-j  )  ace  obtained  by  the  following  operation, 

ski=  mki  /Zj  “ki  >  (3.9) 

The  shape  vector  skj  (s^)  must  satisfy  the  following  proper¬ 
ties  in  addition  to  the  properties  of  normal  Euclidian 
space. 

PROPERTIES  OP  SHAPE  SPACE 

1)  The  shape  vector  is  independent  of  the  size  variable. 

Ski  <  a  Ski’  ”  Ski  ^i1 

where  i  =  1,  2, . ,N,  k=l ,  2,  ....  K, 

and  a  is  a  scalar. 

2)  The  shape  vector  is  independent  of  translation  that  is, 

S(  s  +  s  )  -  S(  8  )  (3.11) 

kj  ki  0  kj  ki 

where  i  =  1,  2  ,  . N,  k=l,  2,  . K, 

and  Sq  is  a  constant  vector. 

3)  The  shape  is  independent  of  rotation. 

S(  a.  +  u  )  =  S(  a.  )  (3.12) 

kj  kj 

where  i=l,  2,  . N  and  cu  is  the  angle  made  by  the  i  th 

component  of  S  with  a  fixed  reference  axis  and  i^is  a 
constant  angle. 

The  vector  obtained  by  using  a  set  of  measurements  on 

a  partial  shape  must  still  be  contained  in  the  space. Unless 
a  size  variable  is  found  which  is  independent  of  both  shapes 
(  both  the  partial  and  the  complete)  it  is  not  meaningfull 


to  compare  the  shape  vector  in  shape  space.  It  is  not  pos¬ 
sible  to  find  a  size  variable  which  is  a  totally  independent 
continious  function  of  measurements  made  on  both  shapes. 
This  being  the  case  the  only  choice  left  is  to  split  the 
shape  into  parts  or  subshapes  and  define  a  size  variable 
which  is  piecewise  continuous  over  these  parts.  Two  theo¬ 
rems  relating  the  subshape  to  the  complete  shape  in  shape 
space  are  stated  and  proved.  These  theorems  are  used  exten¬ 
sively  in  the  sequel. 

THEOREM  1  :  The  vector  formed  by  concatenating  a  series  of 
shape  vector  is  a  shape  vector. 


Proof:  It  is  required  to  show  that  the  shape  vector  result¬ 
ing  from  concatenating  a  series  of  shape  vectors  satisfy  the 
three  properties  of  shape  space. Let  the  concatenated  shape 
vector  be 


cj 


"(Slk  (  Slu  } '  S2 %  (  S2v  5  '  * 


,  S  (  s 
on  ow 


.  -) 


(3.13) 

where  the  first  subscript  on  S  represents  the  object (  shape 

)  and  the  second  represents  the  size  variable  used  to  obtain 

that  shape  vector.  It  should  be  pointed  out  that  k,  ■  and  n 

are  arbitrary  size  variables.  Each  element  of  S  (  s  ) 

cj  ci 

can  be  represented  by 


ci 


=  m_ 


oi 


/zd  (  «cl  ) 


(3.14) 


Therefore, 


r 


s  As  .)  =( 
cj  ci' 


Sci  '  sc2 


c3 


s 


cn 


(3.15) 


Multiplying  each  component  of  (3.13)  by  the  scalar  a  yields, 

a  S  (  s  )  =  (a  S  (s  ),aS  (s  ),...,  a  S  (s  )...) 
cj  ci  lk  lu  21  2v  3m  3w 

(3.16) 


but  from  the  (3.10)  we  have, 


ov 


as  )  =  a  S  (  s 
op  '  ov'  op 


using  (3.17)  in  (3.15)  results  in 


a  «cl)-(a  »cl,a  a^. 
or 


•  t  3  S  t  •  • 

cn 


a  Scj(  sci  >  =  S  a  sci 

CJ 

Equation  (3.19)  proves  property  i) 


(3.17) 

(3.18) 


(3.19) 


To  prove  that  the  shape  vector  satisfies  proverty  ii)  it  is 
only  necessary  to  observe  that  each  member  of  the  concaten¬ 
ated  vector  is  a  shape  vector  is  a  shape  vector . Therefore 
each  satisfies: 

^kj  (  ski  +s0  )=  Skj  (  ski  }  (3.20) 

The  proof  for  property  iii)  follows  in  a  similar  manner 
Q.E.D 


Theorem2:  The  shape  defined  by  a  shape  vector  obtained  by 
concatenating  a  series  of  shape  vectors  is  unique  if  and 
only  if  the  size  variable  defining  each  member  of  the  set  is 
known. 
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Proof : 


A  linear  homogeneous  function  surface  will 


intersect  a  positive  directed  straight  line  at  only  one 
point.  Since  a  shape  is  defined  as  a  point  on  the  shape  ray, 
if  follows  that  any  point  on  this  ray  can  be  uniquely  deter¬ 
mined  by  its  intersection  of  a  size  variable  which  is  de¬ 
fined  as  a  linear  homegeneous  function  over  the  positive 
quadrant. 

:  Assume  otherwise.  Then  there  is  at  least  one  S  (  s 

on  c 

shape  vector  in  the  concatenated  set  whose  size  variable 
z^(n^i)  can  be  choosen  arbitarily.  Then  the  concatenated  vec¬ 
tor  under  this  assumption  would  still  satisfy  the  properties 


of  the  shape  space,  i.e., 
a  Scj  (  sci)  *  Sc.  (  a  sci> 

Now  the  R.H.S  of  (3.21)  can  also  be  expanded  as 


=  (  a  Sik  -  a  & 


,a  Son  •*") 


(3.21) 


(3.22) 


*  (  aM  L 

~zT 


•'a 

z 


)  (3.23) 


but  since  Z  can  be  choosen  arbitrarily  as  long  as  it  sat¬ 
isfies  the  definition  of  a  size  var iable ( 3 .4) .  Choose 

^  ^  <  a  *oi  >  <3-24> 

substituting  (3.4)  and  (3.24)  if  (3.23)  the  following  rela¬ 
tionship  is  obtained 


a  S  .{  s  .  )  =  (  a  M 


,  a  M 


(3.25) 
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which  is  also  equal  to, 


-  (a  slk  »  a  s2z'  son ' a  S0+i,n+r->)  (3*26) 

comparing  (3.22)  with  (3.26)  that, 

a  S  .  (  s  . )  t  S  .(  a  s.) 
cj  v  ci'  r  cj  1  ci' 

which  contradicts  (3.21) 

Q.E.D 

SECTION  IV 
CRITICAL  POINTS 


When  analyzing  shapes  represented  by  sampled  boundries  a 
natural  question  is  one  regarding  the  points  between  which 
the  measurements  should  be  made.  A  simplistic  approach,  an 
approach  which  assumes  that  there  are  the  same  number  N  of 
boundary  points  on  each  shape  and  evades  the  problem  of  ana¬ 
lyzing  shapes  with  a  different  number  of  boundary  points, 
would  be  to  make  measurements  between  all  possible  pairs  of 
boundary  points.  This  leads  to  the  number  NI/(N-2) 12!  of 
measurements  for  each  shape. 

In  order  to  recognize  the  shape,  a  normalization  and  com¬ 
parison  procedure  must  still  be  used.  It  is  obvious  that  for 
relatively  small  N  the  computational  requirements  are  very 
large.  Therefore  it  becomes  necessary  to  consider  measure¬ 
ments  between  a  relatively  few  representative  points  on  the 
shape.  The  chosen  points  on  the  shape  have  to  be  more  impor- 


tant,  in  some  sense,  than  all  the  rest.  The  set  of  points 
which  define  a  shape  may  be  considered  as  a  Fuzzy  set  in 
which  various  points  are  assigned  to  it  with  various  degrees 
of  membership.  Defining  a  precise  criteria  on  which  a  degree 
of  membership  can  be  assigned  to  a  point  on  a  shape  is  very 
difficult.  However,  past  researchers  have  discovered  [2], 
[8],  [18]  that,  the  points  that  should  have  a  higher  degree 
of  membership  than  the  rest  are: 

1)  Points  of  Maxima 

2)  Points  of  Minima 

3)  Points  of  Inflection 

4)  Points  of  Intersection 

5)  End  points  of  open  curves 

6)  Points  where  the  curvature  changes  sign  or  magnitude. 

In  past  shape  analysis  efforts,  points  of  maxima  as  well 
as  minima,  and  inflection  points  of  curves  have  been  ex¬ 
tracted  from  the  shape  data  without  due  consideration  of  the 
coordinate  axes.  This  approach  inevitably  lead  to  errors, 
because  these  points  have  no  meaning  unless  the  coordinate 
axes  are  first  defined.  The  problem  with  such  an  approach 
is  that  if  a  set  of  coordinate  axes  is  chosen  independent  of 
the  shape  under  consideration  then  the  maxima,  minima,  and 
inflection  points  will  not  be  independent  of  rotation  of  the 
shape  with  respect  to  the  coordinate  axes.  The  conclusion 
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is  that  the  coordinate  axis  upon  which  the  maxima,  minima, 
and  points  of  inflection  are  based  must  be  dependent  on  the 
shape  itself. 

The  next  logical  question  is  whether  there  should  be  one 
set  of  coordinate  axes  or  many.  The  answer  is  not  straight¬ 
forward.  Some  shapes  require  more  than  one  set  of  coordi¬ 
nate  axes  while  others  may  require  only  one.  Before  discuss¬ 
ing  methods  that  may  be  used  for  determining  critical 
points,  two  definitions  are  presented  which  will  be  used  in 
the  sequel . 

Definintion  Is  A  curve  c  is  said  to  be  in  line  of  sight  of  a 
point  P  if  every  point  on  the  curve  c  can  be  connected  with¬ 
out  intersecting  the  curve  at  any  other  point.  Otherwise  the 
curve  is  said  to  be  not  in  line  of  sight  of  the  point  P 
(NLSP) .  Examples  of  (LSP)  curves  of  a  point  P,  where  P  is 
the  centroid  of  the  shape  then  denoted  as  (LSC) ,  are  shown 
in  Pig.  10. 

Definition  II:  Line  of  sight  of  a  straight  line  axis  L:  Let 
n  be  the  normal  projection  of  a  curve  c  onto  a  straight  line 
L.  The  curve  c  is  then  said  to  be  in  line  of  sight  of  L  if 
all  points  from  c  can  be  can  be  mapped  injectively  onto  n. 

Examples  of  curves  which  are  in  line  of  sight  of  a  single 
straight  line  axis  (LSA)  are  shown  in  Fig.  11  along  with 
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EXAMPLES  OF  SINGLY  CLOSED  CURVES 
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FIG.  10.  CORVES  DEPICTING  THE  LINE  OF  SIGHT  OF  A  POINT  CONCEPT. 
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FIG.  11.  CORVES  DEPICTING  THE  LINE  OF  SIGHT  OF  AN  AXIS  CONCEPT. 


curves  not  in  line  of  sight  of  a  single  straight  line 
axis (NLSA) 

Dividing  the  shape  into  this  set  of  projections  is  equiva¬ 
lent  to  defining  the  shape  in  terms  of  single  valued  func¬ 
tions.  It  follows  from  the  definition  of  a  single  valued 
function  that  fewer  ambiguities  should  result.  That  is,  the 
critical  points  are  now  determined  from  single  valued  func¬ 
tions  rather  than  multivalued  functions.  The  actual  methods 
for  determining  the  critical  points  can  now  be  presented. 

SECTION  V 

ADAPTIVE  LINE  OF  SIGHT  CRITICAL  POINT  DETERMINATION 

Several  methods  have  been  utilized  in  the  past  for  deter¬ 
mining  critical  points  [2], [8], 118].  Most  of  these  methods 
are  based  on  operations  on  a  fixed  number,  m,  of  adjacent 
points.  Since  it  is  unlikely  that  an  intelligent  machine 
will  have  a  priori  knowledge  about  the  size  of  the  shape  to 
be  analyzed  or  the  relationship  of  the  number  of  sample 
points  to  the  size,  the  ambuiguities  involved  in  detection 
of  critical  points  using  methods  that  are  totally  dependent 
on  operations  on  fixed  number  of  adjacent  points,  are  high. 
The  classical  methods  of  differential  geometry  work  very 
well  on  one  dimensional  data  and  on  theoretical  curves. 
While  each  of  these  methods  has  exhibited  desirable  charac- 
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teristics,  undesirable  characteristics  are  always  present, 
specially  when  the  shape  is  corruptee  by  noise.  A  brief  de¬ 
scription  of  some  of  the  well  known  classical  methods 
[7] , [10] ,  that  work  well  for  theoretical  curves  are  now  pre¬ 
sented  for  the  purpose  of  comparing  their  performace  with 
the  approach  described  in  the  sequel,  namely,  the  Adaptive 
Line  of  Sight  Method.  Also  these  methods  may  be  used  in  the 
second  pass  of  the  this  adaptive  algorithm  after  the  shape 
has  been  rid  of  noise. 

The  method  of  Centroidal  Vectors 
In  this  method  the  ith  point  on  a  shape  at  a  vector  distance 
d^Z-aj^  from  a  reference  point  (  typically  the  shape  centroid 
)  is  said  to  be  critical  with  respect  to  the  reference  point 
if 

((  Idi.J-ldJ  )  and  (  |d.|  -  |di4j|)) 

or 

(  (  I  a  M  a  |  )  and  <  |  ot  |  —  |  a  j  )  ) 

i-1  i  i  i+1 

have  opposite  sign. 

This  operation  is  very  local  in  nature  and  is  extremely 
sensitive  to  noise.  Round-off  or  truncation  errors  also 
have  a  deleterious  effect  on  the  operation.  It  also  fails 
very  often  when  dealing  with  smooth  curves  or  in  situ¬ 
ations  where  the  centroid  is  located  away  from  the  shape 
boundary  as  shown  in  Fig.  12. 
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FIG.  12.  A  CASE  AGAINST  THE  CENTROID 


PIG.  13.  CRITICAL  POINTS  FOR  THE 


Another  method  is  that  of  Curvature  Vectors. 


This 


method  is  basically  a  two  pass  process.  In  the  first  pass, 
the  i  th  point  on  the  shape  with  a  curvature  is  set 

as  a  critical  point  if 
(  <i_i  '  <i  )  or  (<•  -  <i+1  ) 

have  opposite  sign.  In  the  second  pass,  the  points  at  a 
maximum  distance  from  the  straight  line  joining  every  two 
successive  critical  points  is  set  as  critical.  This  meth¬ 
od  like  the  previous  method,  is  very  sensitive  to  noise. 
Though  it  works  for  most  of  the  smooth  curves,  there  are 
instances  where  it  fails.  For  example,  this  method  produc¬ 
es  only  one  critical  point  for  the  cardioid  discussed  in 
section  II. 

In  the  Magnitude  of  Curvature  Method  the  ith  point  is 
determined  to  be  critical  if, 

|  k  |  >  -  Threshold 

The  critical  points  were  determined  by  using  this  method 
on  the  shape  (elephant)  in  Fig.  13.  It  is  apparent  that 
some  critical  points  were  missed.  For  example  the  critical 
points  for  the  trunk  are  missing.  This  procedure  is  more 
immune  to  the  noise  than  the  others;  however  an  equally 
critical  problem  is  added.  Specifically,  a  threshold  must 
be  determined  a  priori.  Additionally  the  method  fails 
completely  on  smoothly  varying  curves  such  as  the  cardioid 
of  Fig.  5. 
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A  procedure  that  performs  well  for  polygonal  shapes 
and  curves  with  very  few  boundary  points  is  called  the 
Line  of  Sight  method.  In  this  method,  if  d'  denotes  the 
normal  distance  of  the  i  th  point  from  a  straight  line  L, 
then  the  i  th  point  on  the  shape  boundary  is  said  to  be 
critical  with  respect  to  the  straight  line  L  if 
(  d.^  -  d.  )  and  (  di  -  d.+1  ) 

have  opposite  sign.  The  set  of  critical  points  found  with 
respect  to  the  set  of  tangent  lines  drawn  at  all  points  on 
the  shape,  will  then  be  called  the  critical  points.  It  is 
obvious  that  for  shapes  other  than  polygonal  an  infinite 
number  of  axes  or  tangent  lines  are  required.  Nonetheless, 
the  attributes  exhibited  by  this  method  are  very  desira¬ 
ble.  It  is  however  necessary  to  reduce  the  dimensionali¬ 
ty  of  the  problem;  this  is  the  subject  that  is  addressed 
by  the  Adaptive  Line  of  Sight  method. 

ADAPTIVE  LINE  OF  SIGHT  METHOD; 

In  the  Adaptive  Line  of  Sight  method  the  critical  point 
determination  is  based  on  a  set  of  coordinate  axes  that 
are  dependent  on  the  shape  under  consideration.  As  previ¬ 
ously  discussed,  this  will  allow  critical  points  to  be  de¬ 
termined  with  fewer  ambiguities.  The  Adaptive  Line  of 
Sight  method  is  an  approach  used  to  emulate  the  Line  of 
Sight  method  described  above  without  having  the  burden  of 


-  86 


excessive  dimensionality.  The  procedure  is  adaptive  in  the 
sense  that  it  adapts  to  the  shape  data  under  considera¬ 
tion. This  will  become  evident  in  the  sequel. 

The  Adaptive  Line  of  Sight  method  is  a  two  pass  pro¬ 
cess.  In  the  first  pass,  the  shape  is  divided  into  a  mini¬ 
mum  number  of  segments  or  parts  by  an  appropriate  set  of 
'critical  points'.  The  members  of  this  segmenting  set  of 
critical  points  are  defined  to  be  those  critical  points, 
such  that  all  boundary  points  in  between  any  two  adjacent 
critical  points  have  the  following  two  properties  with  re¬ 
spect  to  the  straight  line  L  joining  critical  points: 

1) the  boundary  points  are  on  the  same  side  of  a  straight 
line  L  joining  the  adjacent  critical  points; 

2)  the  points  are  in  line  of  sight  of  L. 

From  the  definition  of  line  of  sight  (Definition  II)  it 
is  clear  that  this  means  that  the  boundary  curve  has  a 
unique  one-to-one  projection  on  L.  It  should  be  emphasized 
that  the  minimum  number  of  critical  points  are  obtained 
during  the  first  pass.  This  is  done  by  an  exhaustive 
search  process  that  locates  all  points  such  that  the  above 
properties  are  satisfied.  Often  the  minimum  is  not  uni¬ 
que,  in  which  case,  the  required  set  is  obtained  by  sum¬ 
ming  all  minimum  sets. 
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PLOT  OF  THE  FRONT  PART  OF  THE  SWEPT  WING  PLANE 
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FIG.  IS.  CRITICAL  POINTS  OF  SWEET-WING  PLANE  SHAPES 


In  the  second  pass,  the  points  of  maxima  -  minima  and 
inflection  between  every  two  critical  points  are  detected 
using  the  derivative  of  the  normal  distance  of  the  point 
from  L  with  respect  to  the  distance  along  L.  A  moving  av¬ 
erage  of  these  normal  distance  may  be  used  to  eliminate 
the  effect  of  noise.  The  critical  points  found  in  pass 
one,  the  segmenting  set,  and  the  critical  points  found  in 
pass  two  are  defined  to  be  the  members  of  the  Fuzzy  shape 
set  with  the  highest  degree  of  membership.  A  thorough  de¬ 
scription  of  both  the  computational  and  detection  aspects 
of  the  Adaptive  Line  of  Sight  Algorithm  along  with  a  com¬ 
plete  flowchart,  is  given  in  Appendix  A. 

The  minimal  set  of  critical  points  obtained  by  using 
the  Adaptive  Line  of  Sight  on  the  cardioids  of  Fig.  4,  as 
predicted  are  independent  of  rotation  and  are  shown  in 
Fig.  14.  Fig.  15  shows  a  typical  non  minimal  set  of  crit¬ 
ical  points  obtained  using  the  Adaptive  Line  of  Sight 
method.  Part  a)  of  the  figure  is  the  complete  shape  of  the 
plane,  while  part  b)  shows  the  front  part  of  the  plane 
after  it  has  been  rotated,  displaced  and  reduced  in  size. 
In  spite  of  the  total  dissimilarity,  the  critical  points 
found  for  both  the  partial  plane  shape  and  the  whole  plane 
shape  are  quite  close  as  predicted.  The  clusters  of  cri- 
tial  points  occur  because  of  the  lack  of  a  predefined  res¬ 
olution. 
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SECTION  VI 


FEATURE  SELECTION  &  COGNITIVE  STEP 

The  features  of  a  shape  are  essential  to  defining  the 
shape  in  terms  of  parameters  that  can  be  ultimately  used 
by  machine  for  decision  purposes.  However,  the  manner  in 
which  the  feature  defining  procedure  can  be  selected  is 
quite  variable.  Since  a  dependable  feature  selection  pro¬ 
cedure  is  fundamental  to  the  shape  recognition  problem,  it 
is  essential  that  this  aspect  of  shape  recognition  be  ad¬ 
dressed  with  specificity.  This  point  is  punctuated  when  it 
is  realized  that,  irrespective  of  the  method  of  defining 
and  detecting  critical  points,  a  cognitive  algorithm  is 
still  required  which  examines  in  some  sense,  the  critical 
points  of  the  shape  for  the  purpose  of  reaching  a  decision 
about  some  aspect  of  the  shape.  Consider,  for  example, 
the  shape  shown  in  Fig.  15-a,  a  shape  such  as  this  swept 
wing  plane  may  have  thirty  to  fifty  critical  points.  The 
human  eye  makes  numerous  measurements,  automatically  and 
sub-consciously,  between  the  feature  points  and  determines 
their  relationship  to  one  another.  The  'most  important'  of 
these  measurements  combined  with  the  relationship  between 
them,  comprise  the  decision  set.  The  term  "  most  important 
"  is  difficult  to  define  mathematically,  because  it  is  the 
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result  of  training.  An  unrefined  cognitive  procedure  must 
therefore  consider  the  set  of  all  possible  measurements 
between  the  critical  points.  Obviously  this  is  a  very 
large  number  of  measurements  even  for  numbers  as  modest  as 
thirty  to  fifty.  It  is  well  known  that  observation  that 
this  totality  of  measurements  between  critical  points  is 
not  essential  to  the  decision  process. 

It  is  necessary  therefore  to  determine  methods  for  ac¬ 
quiring  the  minimal  set  of  measurements  or  features  re¬ 
quired  for  the  decision  process.  The  human,  apparently 
places  heavy  weighting  on  features  that  are  formed  by 
critical  points  that  are  symmetrically  opposite  about  an 
axis  and  features  that  are  extracted  from  adjacent  criti¬ 
cal  points  concerning  the  shape.  Without  any  prior  knowl¬ 
edge  a  human  can  find  the  sets  of  axes  about  which  some 
critical  points  are  symmetrically  placed  with  very  little 
effort.  However,  such  a  task  is  almost  insurmountable 
for  a  machine  based  algorithm  unless  it  is  performed  at  a 
post-cognitive  level.  In  the  absence  of  noise,  machine 
recognition  (  cognitive  )  algorithms  perform  reasonably 
well  by  using  only  features  consisting  of  adjacent  criti¬ 
cal  points. 

A  cognitive  algorithm  that  utilizes  measurements  such 
as  these  in  a  continuous  sequential  manner  would  be  en- 
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tirely  adequate  if  the  algorithm  for  detecting  critical 
points  is  totally  immune  to  noise,  round  off,  and  trunca¬ 
tion  errors.  For  example  any  extra  critical  points  that 
are  the  result  of  a  burst  of  noise  would  prevent  any  con¬ 
tinuous  sequential  recognition  algorithm  from  yielding 
conclusive  results. 

One  manner  by  which  this  problem  can  be  circumvented  is 
to  divide  each  of  the  shapes  under  analysis  into  subshapes 
in  terms  of  their  features,  and  then  compare  the  features 
of  these  subshapes  and  the  manner  in  which  they  are  relat¬ 
ed  to  each  other.  It  is  recalled  from  Section  III  that  the 
properties  of  shape  space  dictate  that  the  measurements 
which  define  a  feature  must  be  made  from  either  the  cen¬ 
troid  of  the  set  of  critical  points  or  between  the  criti¬ 
cal  points  that  form  the  feature.  It  is  necessary  then  to 
determine,  in  some  manner,  the  minimum  number  of  critical 
points  that  adequately  define  a  feature.  If  each  feature 
were  defined  in  terms  of  only  two  critical  points,  then 
all  features  would  be  identical  since  the  comparisons  are 
made  with  respect  to  the  same  shape  independent  size  vari¬ 
able.  Therefore,  the  minimum  number  of  critical  points 
that  can  form  a  distinguishable  feature  is  three,  and 
these  must  be  adjacent.  However  unless  the  relationship 
between  these  three  point  features  with  its  adjacent  fea- 


tures  is  also,  considered  any  comparison  (  cognitive  )  al¬ 
gorithm  almost  always  leads  to  ambiguities.  The  reason  for 
this  is  because  the  three  point  feature  forms  a  triangle. 
An  examination  of  the  shape  shown  in  Fig.  15-a  shows  that 
it  contains  many  similar  triangles. 

Unfortunately  the  mathematics  required  to  obtain  the 
optimal  number  of  critical  points  that  should  form  a  fea¬ 
ture  is  not  yet  developed.  Therefore,  it  is  necessary  to 
resort  to  the  psychological  aspects  of  the  human  recogni¬ 
tion  and  decision  process  as  well  as  the  practical  aspects 
such  as  the  implementation  and  computational  requirements. 
These  criteria  lead  to  the  features  being  selected  as  fol¬ 
lows: 

1)  Reconstruct  the  shape  by  connecting  all  the  adjacent 
points  by  a  straight  line.  This  is  called  the  critical 
shape  boundary. 

2) The  feature  F^^  is  then  formed  by  including  critical 
points,  Ci  and  at  most  three  adjacent  critical  points  on 
each  side  of  The  critical  points  chosen  must  be  in 
line  of  sight  of  .  This  means  that  it  must  be  possible 
to  draw  a  straight  form  to  each  of  the  other  critical 
points  defining  the  feature  without  intersecting  the  crit¬ 
ical  shape  boundary.  The  feature  obtained  by  this  proce¬ 
dure  cannot  be  defended  as  optimal  in  any  mathematical 
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sense.  However,  it  correlates  quite  well  with  those  enti¬ 
ties  that  humans  consider  features.  The  features  corre¬ 


sponding  to  critical  points  and  C-^are  shown  in  Fig. 
16. 

A  desirable  improvement  to  the  above  feature  defining 
procedure  is  an  algorithm  for  deciding  whether  the  line 
joining  C  i  to  another  critical  point  in  the  feature  lies 
inside  or  outside  the  shape. 

The  cognitive  step  requires,  as  usual  a  dictionary  of 
the  features  of  the  complete  shape  against  which  the  par¬ 
tial  shapes  are  to  be  compared.  The  partial  shape  dic¬ 
tionary  will  henceforth  be  referred  to  as  the  problem 
text.  One  page  of  the  complete  shape  dictionary  is  shown 
in  Table  1.  This  page  contains  features  twenty-one  and 
twenty-two  of  the  swept  wing  plane  of  Fig.  17.  The  table 
includes,  in  addition  to  the  feature  number,  the  critical 
points  of  that  feature  along  with  their  x  and  y  location, 
the  x  and  y  location  of  the  centroid  of  all  the  critical 
points  contained  in  the  feature,  the  size  of  the  feature, 
the  normalized  components  of  the  shape  vector,  and  the 
normalized  angle  of  the  shape  vector  component. 

The  normalized  shape  vector  component  are  defined 

where  s  is  obtained  by  the  relationship, 

-  2  -2 

8ki  sqrt  ((xki~x)  +  (yki"  Y))/  Zj  (6,1) 
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where  the  size  variable  was  chosen  to  have  the  form, 
n 


Z^=  i  abs(xkr  x)  +  abs(y^-  y) 


i-1 


ki 


(6.2) 


and  the  subscript  is  the  feature  number.  The  normalized 
component  of  the  angle  of  the  shape  is  obtained  in  a  simi¬ 
lar  fashion.  These  same  quantities  are  obtained  to  form  a 
dictionary  for  the  partial  shape  (problem  text) .  Each 
page  of  both  the  dictionaries  begins  with  a  feature  set 
that  is  not  a  subset  of  another  feature  set.  This  feature 
set  is  defined  as  the  uncovered  feature  set.  The  covered 
feature  sets  are  arranged  in  the  order  of  cardinality  be¬ 
low  the  uncovered  feature  set  on  each  page.  The  purpose  of 
this  architecture  is  to  simplify  the  computational  re¬ 
quirements  for  the  cognitive  step. 


In  general  all  the  features  in  the  dictionary  will  not 
be  contained  in  the  problem  text.  It  is  also  true  that  the 
problem  text  contains  features  that  are  not  present  in  the 
dictionary.  This  becomes  apparent  by  examining  Fig.  9. 
Therefore,  the  fact  that  a  feature  is  contained  in  the 
problem  text,  does  not  imply  that  the  partial  shape  is  not 
a  part  of  the  complete  shape,  because  it  is  not  necessary 
for  the  partial  shape  to  have  fewer  points  that  the  whole 
shape.  Therefore,  further  examination  is  required  before 
a  decision  regarding  the  problem  text  can  be  made. 
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TABLE  1 


A  PAGE  OF  THE  DICTIONARY  SHOWING  FEATURES  21-4  22 


CRITICAL  POINT _ COORDINATE _ FEATURE  VECTOR 


SEQUENCE  # 

X- LOCATION 

Y -LOCATION 

DISTANCE 

NORM.  ANGLE 

C1 

*k.  i 

mm 

V1 

'V.i 

18 

460 

330 

0.095814 

-.0666 

19 

475 

335 

0.121843 

-.0746 

20 

475 

350 

0.111665 

-.0538 

21 

450 

350 

0.047611 

-.0278 

22 

410 

385 

0.098166 

-.0650 

23 

385 

385 

0.145874 

-.0964 

24 

370 

345 

0.162900 

-.0615 

FEATURE  9  k  -  21 

SIZE  -  385.71  ,  ANG  MEAN  -  .6889 

CENTROID  COORDINATES 
X  -  432.143  ,  Y  -  354.286 


20 

475 

350 

0.219780 

0.  1884 

21 

450 

350 

0.129848 

0.  0269 

22 

410 

385 

0.088005 

-0. 6084 

23 

385 

385 

0.149101 

-0.  1748 

24 

370 

345 

0.192721 

0.  7716 

FEATURE  if  k  -  22 

SIZE  -  266  :  ANG  MEAN  -  -.4126 

CENTROID  COORDINATES 
X  •  418'  ,Y  *  363 


TABLE  2 


A  PORTION  OF  THE  PROBLEM  TEXT  SHOWING  FEATURES  12  A  13 


CRITICAL  POINT _ COORDINATE  FEATURE  VECTOR 


SEQUENCE  It 

X-LOCAT ION 

Y-LOCATION 

DISTANCE 

NORM.  ANGLE 

Ci 

\,1 

^k,  i 

Sk,i 

“k.i 

9 

2.8429 

11.1716 

0.096285 

-.0644 

10 

2.9844 

11.4543 

0.122441 

-.0701 

11 

2.7724 

11.6666 

0.112214 

-.0485 

12 

'  2.4187 

11.3131 

0.047845 

-.0258 

13 

1.3580 

11.2429 

0.098648 

-.0646 

14 

1.0043 

10.8894 

0. 146590 

-.0928 

15 

1.3575 

10.1115 

0.163700 

-.0513 

FEATURE  it  k  *  12 

SIZE  -  7.6766  ,  ANG  MEAN  -1.38 

CENTROID  COORDINATES 
X  -  2.1054  ,  Y  -  11.1213 


11 

2.7724 

11.6666 

0.060616 

-0.0535 

12 

2.4187 

11.3131 

0.037482 

0.0308 

13 

1.3580 

11.2429 

0.037192 

0.0596 

14 

1.0043 

10.8894 

0.043854 

0.0878 

15 

1.3575 

10.1115 

0.037192 

0.0463 

1 

2.6298 

8.8382 

0.091575 

0. 1094 

FEATURE  if  k  -  13 

SIZE  -  21.5099  ,  ANG  MEAN  -  0.13964 

CENTROID  COORDINATES 
X  ■  1.92349  ,  Y  -  10.5769 
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The  decision  procedure  consists  of  selecting  an  arbi¬ 
trary  word  from  the  problem  text  dictionary.  A  problem 
text  word  is  of  course  a  feature  from  the  partial  shape 
under  comparison.  The  shape  vector  from  the  problem  text 
is  compared  to  the  shape  vector  in  the  dictionary  always 
starting  on  page  one  of  the  dictionary.  The  comparison 
continues  until  a  match  occurs.  The  next  step  is  to  com¬ 
pare  the  next  problem  text  feature  vector  in  order  of  , 
cardinality,  to  the  next  feature  in  the  dictionary  and  so 
forth.  An  example  of  this  technique  is  given  by  comparing 
Tables  1  and  2.  In  this  experiment  a  feature  vector  from 
the  partial  shape  was  selected  for  comparison.  It  should 
be  emphasized  that  the  feature  vector  is  from  the  problem 
text  of  the  partial  swept  wing  aircraft  shown  in  Fig.  16. 
The  partial  shape  has  been  rotated  and  shifted  as  well  as 
scaled  to  insure  that  any  direct  template  matching  proce¬ 
dure  will  fail.  This  also  demonstrates  that  the  concaten¬ 
ated  feature  vector  matching  procedure  described  here  is 
independent  of  rotation,  size  and  location.  Feature  vector 
twelve  (  word  )  was  arbitrarily  selected  from  the  problem 
text.  By  comparing  tables  1  and  2  it  is  apparent  that  the 
word  12  matched  feature  vector  21  of  the  dictionary.  It 
should  be  noted  that  this  match  occurs  even  though  the  lo¬ 
cation  of  the  critical  points  and  the  centroid  of  the  fea¬ 
ture  of  the  partial  shape  are  different  from  those  same 


quantities  for  the  whole  shape  because  of  the  rotation  and 
shift.  In  this  way  a  correspondence  table  is  then  estab¬ 
lished  between  the  critical  points  of  the  features  in  the 
dictionary  to  the  critical  points  of  the  word  in  the  prob¬ 
lem  text  The  next  step  is  to  proceed  in  sequential  order 
to  the  next  word  in  the  problem  text  e.g.  word  13  (n+1) 
which  is  sequentially  next  to  word  12  (n)  of  the  problem 
text  does  not  match  feature  vector  22  (m+1)  of  the  dic¬ 
tionary.  However  since  feature  vector  22  is  on  the  same 
page  as  feature  vector  21  (  m  )  of  the  dictionary  word  13 
is  mismatched  to  feature  22  because  it  contains  a  critical 
point  which  is  not  contained  in  word  12.  An  examination 
of  tables  1  and  2  indicates  that  the  critical  point  C-^  ' 
is  contained  in  word  13  but  not  in  word  12. 

The  mismatched  critical  point  is  first  compared  tc  t.v*4 
correspondence  table.  If  it  is  not  found  in  the  correspon¬ 
dence  table  then  it  is  stored  in  a  mismatch  table.  At  any 
latter  stage  a  mismatched  critical  point  is  erased  from 
the  mismatched  table  if  some  word  containing  the  mis¬ 
matched  critical  point  matches  some  feature  of  the  dic¬ 
tionary. 

In  general  if  word  n  matches  feature  m  then  it  is  ex¬ 
pected  that  word  {  n  +1  )  will  match  feature  (  m  +  1) .  If 
feature  (m+1)  is  on  the  same  page  as  feature  m  then  it  is 
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TABLE  3 


REVISED  FEATURE  VECTOR  #  22 


CRITICAL  POINT  _ COORDINATE _ FEATURE  VECTOR 


SEQUENCE  # 

X-LOCATION 

Y-LOCATION 

DISTANCE 

NORM.  ANGLE 

Ci 

Xk,i 

•H 

mem 

ak,i 

20 

475 

350 

i 

0.275903  ! 

0.3374 

21 

450 

350 

0.108797 

-.4576 

22 

410 

385  1 

.  - - - -J 

0.360555 

0.1203 

FEATURE  #  k-22 
SIZE  -  116.667 
X-CENTROID-445 


REVISED  TO  D 

ANG  MEAN  -  -0.7827 
Y-CENTROID  -  361.667 


21 

450 

— 

350 

— 

0.360555 

0.1203 

22 

410 

385 

0.108797 

-.4576 

23 

385 

385 

0.275903 

0.3374 

FEATURE  //  k-22 
SIZE  -  116.667 
X-C  ENTRO ID “4 1 5 


REVISED  TO  D 

ANG  MEAN  -  -0.7827 
Y-CENTROID  -  373.333 


22 

410 

385 

0.263178 

0.4869 

23 

385 

385 

0.142178 

-1.3906 

24 

370 

345 

0.334767 

0.9037 

FEATURE  #  k-22 
SIZE  -  96.667 
X-CENTROID-388 


REVISED  TO  D 

ANG  MEAN  -  .06478 
dY -CENTROID  -  371.667 
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TABLE  4 


r 


CRITICAL  POINT _ COORDINATE  _ FEATURE  VECTOR 


SEQUENCE  if 

X-LOCAT ION 

— 

Y-LOCATION 

DISTANCE 

NORM.  ANGLE 

c. 

1 

Xk.i 

Yk,i 

Sk,i 

ak,i 

11 

2.7724 

11.6666 

0.296925 

0.3375 

12 

2.4187 

11.3131 

0.117094 

-.4578 

13 

1.3580 

11.2429 

■ 

0.388017 

0.1203 

FEATURE  it  k  «  13  REVISED  FEATURE  TO  P  =  13,1 
SIZE  -  2.6183  ANG.  MEAN  -  0.07666 
X -CENTROID  -  2.18303  Y-CENTROID  *  11.4075 


12 

2.4187 

11.3131 

0.388017 

0.1203 

13 

1.3580 

11.2429 

0.117094 

-.4578 

14 

1.0043 

10.8894 

0.296925 

0.3375 

FEATURE  it  k  -  13  REVISED  FEATURE  TO  P  -  13,2 
SIZE  -  2.6182  ANG  MEAN  -  0.07666 
X-CENTROID  -  1.5936  Y-CENTROID  -  11.1485 


13 

1.3580 

11.2429 

0.291751 

1.5340 

14 

1.0043 

10.8894 

0.157579 

-.3433 

15  j 

1.3575 

10.1115 

0.371073 

-1.1907 

FEATURE  it  k  =  13  REVISED  FEATURE  TO  P  -  13,3 
SIZE  -  1.74413  ANG  MEAN  -  -.19739 
X-CENTROID  -  1.23993  Y-CENTROID  -  10.7479 


14 

1.0043 

10.8894 

0.277421 

-.1909 

15 

1.3575 

10.1115 

0.083901 

0.2752 

1 

2.6298 

8.8382 

0.354384 

_ _ 

-.0843 

FEATURE  it  k  -  13  REVISED  FEATURE  TO  P  -  13,3 
SIZE  -  4.14820  ANG  MEAN  -  -.76957 
X-CENTROID  -  1.6638  Y-CENTROID  -  9.9464 
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easy  to  isolate  the  mismatched  point  as  in  the  above  exam¬ 
ple.  If  feature  m+1  is  not  on  the  same  page  as  feature  m 
or  the  concept  of  pages  is  not  used  then  in  order  to  iso¬ 
late  the  mismatched  critical  point  then  the  feature  m+1 
and  the  word  n+1  have  to  be  revised  into  concatened  shape 
vectors  with  each  subshape  vector  of  three  measurements. 
A  revised  feature  vector  for  feature  22  is  shown  in  Table 
3.  While  the  revised  words  for  word  13  are  shown  Table  4. 
A  comparison  of  the  above  two  tables  again  isolates  as 
the  mismatched  critical  point.-  It  may  also  be  noted  from 
tables  that  angle  being  a  multivalued  function  is  an  unre¬ 
liable  variable  for  comparison.  At  the  end  of  this  com¬ 
parison  process  an  intelligent  machine  would  compare  other 
shape  parameters  like  curvature,  symmetry  etc.  It  may  al¬ 
so  try  to  investigate  if  the  mismatched  critical  points  are 
due  to  noise  errors,  or  due  to  changes  in  the  shape. 
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CONCLUSION 


In  this  paper,  it  is  shown  by  example  that  several  of  the 
previously  described  shape  algorithms  do  not  perform  well  on 
arbitrary  shapes.  While  each  of  the  methods  exhibit  specific 
attributes,  it  is  relatively  easy  to  find  shapes  that  render 
each  of  the  algorithms  useless.  This  is  particularly  true 
when  the  shapes  under  comparison  are  partial  shapes. 

In  particular  an  example  was  presented  that  demonstrated 
that  the  Fourier  Descriptors  method  is  not  suitable  as  a 
general  method  for  the  recognition  of  partial  shapes.  It  was 
also  demonstrated  that  the  peaks  obtained  by  the  Local 
descriptors  are  not  independent  of  rotation. 

In  the  process  of  defining  a  shape  recognition  procedure 
that  would  overcome  or  circumvent  the  weaknesses  of  these 
algorithms,  the  basic  requirements  for  the  shape  recognition 
process  were  stated.  A  new  concept  of  treating  shapes  as 
vector  in  shape  space  was  introduced  and  described.  Also  two 
theorems  relating  to  the  process  of  comparing  partial  shapes 
to  the  complete  shape  were  stated  and  proved. 

A  new  proceedure  of  determining  the  critical  points  of  a 
shape  was  described.  This  procedure  is  named  the  Adaptive 
Line  of  Sight  method.  In  the  Adaptive  Line  of  Sight  method  , 
the  critical  point  determination  is  based  on  a  set  of 
coordinate  axes  that  are  dependent  on  the  shape  being 
examined.  Examples  were  given  that  demonstrate  that  the 


procedure  produces  critical  points  that  are  independent  of 
rotation,  size,  displacment,  and  correspond  closely  to  those 
produced  by  normal  human  cognitive  process.  The 
experimental  results  indicate  that  this  algorithm  ,  like  the 
human  eye,  converges  to  a  minimal  set  of  critical  points 
called  the  segmenting  set. 

The  minimal  segmenting  set  found  by  this  method  in  rare 
instances  does  not  coincide  with  the  minimum  segmenting  set 
found  visually  but  the  critical  points  turn  out  to  be  the 
same.  For  instance,  a  shape  that  is  more  than  a  half-circle, 
but  less  than  a  full  circle  will  lead  to  ambiguities. 

It  was  demonstrated  that  measurements  between  a  set  of 
adjacent  critical  points  that  were  determined  by  using  this 
method  can  be  used  to  define  feature  vectors  for  shapes. 
These  feature  vectors  remain  the  same  whether  the  shape  is  a 
partial  shape  or  a  more  complex  whole  shape.  Moreover,  these 
feature  vectors  are  independent  of  size,  rotation  and 
displacement  since  they  are  derived  from  a  set  of  critical 
points  that  are  independent  of  the  same  quantities.  A 
technique  for  comparing  the  feature  vectors  of  a  set  of 
shapes  is  described.  The  comparison  procedure  is  based  on  a 
syntactic  technique  which  will  point  out  whether  the  shapes 
are  the  same,  whether  one  shape  is  part  of  a  more  complex 
whole  shape,  or  whether  the  shapes  are  totally  dissimilar. 
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FLOWCHART  FOR  THE  ADAPTIVE  LINE  OF  SICHT  ALGORITHM 


FIG  18,  FLOWCHART  FOR  THE  ADAPTIVE  LINE  OF  SIGHT  METHOa 
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Appendix  A 


Refering  to  the  flow-chart  shown  in  Fig.  18,  Side  A  refers 
to  the  condition  when  the  computations  are  performed  from  I 
to  J  modulo  the  number  of  points  in  the  shape,  while  side  B 
refers  to  the  condition  when  computations  are  performed  from 
J  to  to  I.  Forward  track  or  Ftrack  denotes  to  a  condition 
when  I  is  held  constant  while  J  is  incremented.  Backtrack 
or  Btrack  denotes  a  condition  when  J-l  is  held  constant 
while  I  is  decremented. 

Initially  I  and  J  are  always  chosen  to  be  adjacent 
points, first  going  in  the  clockwise  direction  then  in  the 
clockwise  direction  The  details  of  the  computational  block 
and  the  detection  block/process  are  as  follows 
COMPUTATIONAL  BLOCK 

Find  the  equation  of  the  straight  line  S  joining  I  to  J 
Find  the  distance  DISTAIJ  between  points  I  and  J. 

Find  the  equations  of  the  straight  lines  normal  to  L  and 
joining  every  point  P  in  between  I  and  J. 

Find  the  intersection  (XINTSEIJ  ,  YINTSEIJ)  of  each  of 
the  normal  lines  found  in  the  above  step  with  the  straight 
line  L 

Find  the  distances  DISTXIIJ  and  DISTYIIJ  from  point 


i  I  to  the  intersection  and  from  point  J  to  the  intersection, 

respectively  for  every  P. 

Find  the  normal  distance  NDISTAIJ  from  every  P  to  the 
straight  line  L. 

Find  the  normal  vectors  from  the  straight  line  to  every 
point  P. 

DETECTION  PROCESS 

In  this  block  a  detect  switch  is  set  indicating  that  a  crit¬ 
ical  point  has  been  detected  at  I  and  J-l,  if  at  the  first 
instance,  a  point  is  found  which  is  not  on  the  same  side  of 
L  as  other  points,  or  a  point  cannot  be  mapped  injectively 

i 

on  to  the  straight  line  L.  The  former  condition  is  checked 
'  by  comparing  the  magnitude  of  the  sum  of  every  two  adjacent 

normal  vectors  with  the  magnitude  of  those  forming  the  sum  ± 
a  THRESHOLD1,  while  the  latter  condition  is  checked  by  com¬ 
paring  the  sum  of  the  distance  DISTXIIJ+DISTYIIJ  to  DISTAIJ 
+  THRESH0LD2 .  Where  thresholds  1  and  2  are  set  to  account 
for  round-off,  truncation,  quantization  and  other  errors. 
SWITCH  BOARD 

This  is  a  control  block  which  forces  the  computations  to  oc¬ 
cur  in  an  alternating  sequence  FORWARD-TRACK  -BACK-TRACK 
-FORWARD-  TRACK . 
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